Chip placement collaborative control system for flexible production

Abstract

The present invention discloses a chip placement collaborative control system for flexible production, comprising a perception module, a decision module, and an execution module. The system utilizes an improved Radon transform and polar coordinate screening, and employs dual-threshold segmentation and background compression to reduce noise interference and enhance feature extraction robustness. Multi-resolution chip positioning enables precise positioning of the chip within the machine coordinate system. An improved APF-RRT algorithm combines the artificial potential field's attraction-repulsion field with RRT random search to dynamically plan collision-free paths, optimizing path length, time, and safety. Master-slave collaborative control maintains collaborative spacing between devices, adapting to flexible production requirements. Quintic polynomial interpolation is used to generate smooth acceleration curves, reduce mechanical vibration, and improve placement stability. A closed loop of visual positioning, path planning, and dynamic control is formed, enabling adaptive adjustment of flexible production. This results in closed-loop management of chip placement production with higher precision, faster response, and greater adaptability than traditional methods.

Description

Technical Field

[0001] The present invention relates to the technical field of chip placement collaborative control, and in particular to a chip placement collaborative control system for flexible production. Background Art

[0002] Chip placement is a critical process in electronics manufacturing, requiring equipment capable of operating with extremely high precision and speed. Each chip and electronic component must be precisely positioned to prevent problems such as short circuits and open circuits, impacting product reliability and performance. A single assembly line model often cannot cope with complex production tasks under flexible production conditions, necessitating the introduction of more intelligent control methods and coordination mechanisms. In a flexible production environment, collaborative control systems can dynamically adjust based on real-time data to address uncertainties such as production plans and product model changes.

[0003] Currently, Chinese invention patent application number CN202010183406.3 discloses a mathematical model-based motion control method and system for a chip placement machine. The method obtains a static friction curve of a work head; determines nonlinear friction based on the static friction curve; constructs a mathematical model of the chip placement machine's motion based on dynamic equations and nonlinear friction; and controls the movement of the chip placement machine based on the mathematical model. Controlling the motion of a chip placement machine using the above-described method of the present invention can improve the accuracy of chip placement by the chip placement machine. However, existing technologies cannot achieve sub-pixel fiducial center positioning using parabolic fitting, cannot implement multi-resolution chip positioning for precise positioning of the chip in the machine coordinate system, cannot dynamically plan collision-free paths while optimizing path length, time, and safety, cannot maintain coordinated spacing between devices through master-slave collaborative control, and lacks three-dimensional dynamic operation coding to generate a three-dimensional trajectory path with Z-axis height. Consequently, they cannot form a dynamic closed loop from visual positioning (perception) → path planning (decision-making) → dynamic control (execution) to achieve adaptive adjustment for flexible production. Summary of the Invention

[0004] The technical problem solved by the present invention is: the existing technology cannot combine parabola fitting to achieve sub-pixel level reference point center positioning, cannot achieve multi-resolution chip positioning to achieve precise positioning of the chip in the machine coordinate system, cannot dynamically plan a collision-free path, and simultaneously optimize the path length, time and safety, cannot maintain the collaborative spacing between devices through master-slave collaborative control, and does not have three-dimensional dynamic operation coding to generate a three-dimensional trajectory path with Z-axis height. It is impossible to form a dynamic closed loop of visual positioning (perception) → path planning (decision-making) → dynamic control (execution) to achieve adaptive adjustment of flexible production.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a chip placement collaborative control system for flexible production, comprising a perception module, a decision module, and an execution module:

[0006] The perception module is used to collect PCB and chip images through an industrial camera and locate the chip reference point position through machine vision;

[0007] The decision module is used for path planning and coordinated control;

[0008] The execution module is used for multi-dimensional collaborative operations.

[0009] Preferably, the sensing module includes a PCB positioning unit and a chip positioning unit:

[0010] The PCB positioning unit includes locating a cross-shaped reference point on the PCB board and establishing a global coordinate system. The establishment of the global coordinate system specifically includes: using piecewise linear transformation to enhance the contrast of the image collected during the chip placement process, using Gaussian filtering to suppress image noise and retain edge details, extracting the reference point contour through Canny edge detection, improving Radon transformation to detect the straight line feature of the cross-shaped reference point, screening candidate lines through polar coordinate transformation, and fitting the reference point center in combination with the least squares method;

[0011] The chip positioning unit includes: quickly locating the chip area under low resolution, refining the angle matching under high resolution, extracting chip pin features through Harris corner detection based on the point set registration algorithm, constructing the transformation matrix between the ideal model and the actual point set, and combining rotation, translation and scaling parameters to calculate the chip's position matrix relative to the machine coordinate system.

[0012] Preferably, the PCB positioning unit specifically includes:

[0013] Dynamically divide the grayscale interval, linearly stretch the reference point area, compress the background grayscale interval, and optimize the edge reference points through dual thresholds. The optimization process includes:

[0014] Set the grayscale threshold interval. Edge grayscale points above the grayscale threshold interval are considered strong edges, and edge grayscale points below the grayscale threshold interval are considered weak edges. Only weak edges connected to strong edges are retained. The contours of all reference points are extracted. Parabolic fitting is performed on the discrete edge points detected by Canny. The sub-pixel edge position is calculated to obtain a sub-pixel edge image.

[0015] The sub-pixel edge image is projected into a polar coordinate system, and the grayscale integral projection at each angle in the polar coordinate system is calculated. The cross-edge lines are detected and deleted by Hough transform, and the remaining candidate lines after deleting the cross-edge lines are grayscale fitted, and the calculated intersection of the grayscale fitting is used as the reference point center.

[0016] Preferably, the chip positioning unit specifically includes:

[0017] A multi-resolution positioning strategy is used to quickly screen candidate chip areas in the global image. When the resolution is low, the image is downsampled and morphological operations are used to extract the approximate chip outline, including dilation and erosion.

[0018] When in high-resolution condition, the approximate chip outline is refined into a candidate chip area, and local feature extraction is performed. The local feature extraction includes: extracting chip pin feature points in the candidate area using Harris corner detection.

[0019] Preferably, extracting chip pin feature points specifically includes:

[0020] Calculate the gradient of each benchmark pixel point in the refined candidate chip area, and construct the gradient covariance matrix of each benchmark pixel point based on the gradient information. The mathematical expression of the covariance matrix is:

[0021] ;

[0022] in, and are the gradients of the image in the x and y directions, respectively, where x and y are the reference pixels in the image;

[0023] Calculate the Harris corner response value, the mathematical expression of the Harris corner response value is:

[0024] ;

[0025] Wherein, k is an empirical constant, and its value is [0.04, 0.06];

[0026] Filter the reference pixel points whose corner response values ​​are higher than the preset response threshold, retain the gradient maximum point, and perform eigenvalue decomposition based on the basic matrix calculation on the covariance matrix to obtain the eigenvalue and , build an ideal chip model and output the chip pin feature points.

[0027] Preferably, constructing an ideal chip model includes:

[0028] Perform ICP iterative closest point matching based on the chip pin feature points and the pre-extracted actual point set, calculate the rotation matrix, translation vector and scaling factor, and generate a pose matrix;

[0029] The mathematical expression of the pose matrix is:

[0030] ;

[0031] Among them, N is the pose matrix, R is the rotation matrix, T is the translation vector, and S is the scaling factor;

[0032] The chip's local coordinate system is converted to the machine coordinate system to obtain the actual position and orientation of the chip. The reference point coordinates in the machine coordinate system, the rotation angle of the chip relative to the machine coordinate system, and the chip pose matrix are output. The reference point coordinates are the chip pin feature points.

[0033] Preferably, the decision module includes a planning unit and a control unit:

[0034] The planning unit includes:

[0035] Generate a multi-dimensional collaborative operation path based on the chip pin feature points, perform non-repetitive coding on the chip placement machine equipment and obtain the operation code, and build the mechanical equipment connecting rod coordinate system based on the operation code;

[0036] Generating a multi-dimensional collaborative operation path based on the chip pin feature points specifically includes:

[0037] The chip pin feature points are randomly extracted as path nodes, and a collision-free path is generated using the RRT algorithm. The chip placement operation path is optimized for multiple objectives, including path length, time, and safety.

[0038] Obtaining the job code includes:

[0039] Generate time series path points based on the placement machine's beat cycle, superimpose the Z-axis height coordinate information of the chip pin feature point machine coordinate system on the path of the discrete time series path points to generate a three-dimensional trajectory path, assign a code prefix corresponding to each process stage according to the process stage, which includes loading, placement and inspection, and output a three-dimensional dynamic operation code;

[0040] Constructing the mechanical equipment link coordinate system includes:

[0041] An adaptive mechanical device coordinate system is constructed through forward kinematics and inverse kinematics to control the position and posture of the end effector of the mechanical device. The position and posture of the end effector are calibrated and annotated in the mechanical device coordinate system. The position and posture of the end effector of the mechanical device are obtained by chain multiplication of the link transformation matrix. The joint angles of the end effector of the mechanical device are optimized through the particle swarm algorithm, and precise control is achieved in combination with the Jacobian matrix constraint.

[0042] Preferably, the control unit is used to perform master-slave collaborative formation control on the multi-dimensional collaborative operation path:

[0043] Master-slave collaborative formation control includes global path planning of the master device and tracking feedback of the slave devices;

[0044] The master device's global path planning includes: inputting the chip placement task sequence and the global map, using the improved APF-RRT algorithm for path planning, constructing the gravitational field of the target point and the repulsive field of obstacles, and generating a collision-free and smooth optimized path;

[0045] Path planning using the improved APF-RRT algorithm includes:

[0046] S01: Initialize the optimization environment and set global path parameters, which include the initial joint coordinates, angles, and obstacle information of the end effector of the mechanical device. The obstacle information includes the coordinate set of the mechanical device link coordinate system that the end effector of the mechanical device is prohibited from reaching. Set the optimization target to the expected joint coordinates.

[0047] S02: Initialize global path planning based on the multi-dimensional collaborative operation path;

[0048] S03: Calculate the resultant force on the end effector of the current mechanical device. The resultant force is the resultant force of all forces obtained by force analysis according to Newton's law of mechanics. Set a resultant force threshold F_min. When the resultant force is greater than the resultant force threshold F_min, it indicates that the current path planning is valid. Continue to detect the resultant force in real time and execute step S05.

[0049] When the resultant force is less than the resultant force threshold F_min, it indicates that the current path planning is invalid, and step S04 is executed;

[0050] S04: Perform a limited search using the improved RRT algorithm, select the node on the RRT growth tree closest to the current key angle as the temporary target joint angle, and continue to execute step S02;

[0051] S05: Determine whether the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point: when the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point, the path planning optimization task is completed and the loop is exited;

[0052] When the joint angle of the current end effector of the mechanical device does not reach the expected coordinate of the target joint point, return to step S02.

[0053] Feedback from device tracking includes:

[0054] A feedback linearization control architecture is used to perform error compensation based on the Jacobian matrix. The six-dimensional state equation is established through the LQR controller and the optimal control law is solved to achieve accurate trajectory tracking. A virtual spring-damper model is used to maintain the desired spacing between master and slave devices. A distributed consensus algorithm is used to synchronize device states in the communication network.

[0055] Preferably, the execution module includes:

[0056] Based on PID control theory, a three-loop controller of position, velocity and acceleration is designed. An error model is constructed according to the multi-dimensional collaborative operation path output by the decision module. The S-shaped curve optimizes the acceleration and deceleration process of the motion trajectory. The chip position deviation is continuously monitored through the visual system, and the fuzzy adaptive control is used to adjust the placement head posture. The online calibration algorithm is used to compensate for the visual system drift caused by environmental changes.

[0057] Building an error model involves:

[0058] Define the errors of position, velocity and acceleration, and obtain the real-time actual position of the chip based on the mechanical equipment connecting rod coordinate system output by the decision module. Compare the coordinates with the preset target position to obtain the error vector.

[0059] Preferably, the S-curve acceleration and deceleration optimization includes:

[0060] Acceleration planning generates acceleration curves through quintic polynomial interpolation method, constraining the initial / final velocity, acceleration and chip displacement continuity.

[0061] The PCB reference points and chip pin feature points are updated according to the preset cycle, the offset of the mechanical equipment link coordinate system is calculated, the calibration error model is fitted using the RANSAC algorithm, and the impact of the equipment operation time and temperature on the visual system is calculated in real time. The calibration error is fed back as an error function to the multi-dimensional collaborative operation path, and the position and posture of the end effector are corrected through master-slave collaborative formation control.

[0062] The beneficial effects of the present invention are as follows: the improved Radon transform and polar coordinate screening are combined with parabola fitting to achieve sub-pixel level reference point center positioning, which is 10 times more accurate than traditional methods. Through dual threshold segmentation and background compression, noise interference is reduced and the robustness of feature extraction is enhanced. Multi-resolution chip positioning enables precise positioning of the chip in the machine coordinate system. The improved APF-RRT algorithm combines the gravitational-repulsive field of the artificial potential field with RRT random search, dynamically plans collision-free paths, optimizes path length, time, and safety, and uses master-slave collaborative control to maintain collaborative spacing between devices. Three-dimensional dynamic operation coding generates a three-dimensional trajectory path with Z-axis height to adapt to flexible production needs. Quintic polynomial interpolation is used to generate a smooth acceleration curve, reduce mechanical vibration, and improve patch stability. The calibration error is fitted through the RANSAC algorithm, and the influence of environmental factors such as temperature and mechanical drift are compensated in real time. Visual positioning (perception) → path planning (decision-making) → dynamic control (execution) forms a closed loop to achieve adaptive adjustment of flexible production. Combining technologies from multiple fields such as machine vision, robotics, optimization algorithms, and control theory, it breaks through the limitations of a single method, is suitable for micron-level patch needs, supports multi-variety and small-batch production, and adapts to the trend of intelligent manufacturing. The dynamic error compensation mechanism copes with environmental interference and improves the yield rate. Through three major innovations: vision-enhanced positioning, intelligent collaborative control, and dynamic error compensation, the system achieves closed-loop management of chip placement production with higher precision, faster response, and greater adaptability than traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the basic flow of a chip placement collaborative control system for flexible production provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0065] Reference Figure 1 , which is an embodiment of the present invention, provides a chip placement collaborative control system for flexible production, including a perception module, a decision module, and an execution module:

[0066] The perception module is used to capture PCB and chip images through industrial cameras and locate the chip reference point position through machine vision;

[0067] The decision module is used for path planning and collaborative control;

[0068] The execution module is used for multi-dimensional collaborative operations.

[0069] The sensing module includes a PCB positioning unit and a chip positioning unit:

[0070] The PCB positioning unit locates the cross-shaped reference point on the PCB board and establishes a global coordinate system. The establishment of the global coordinate system specifically includes: using piecewise linear transformation to enhance the contrast of the image collected during the chip placement process, using Gaussian filtering to suppress image noise and retain edge details, extracting the reference point contour through Canny edge detection, improving Radon transform to detect the straight line features of the cross-shaped reference point, screening candidate lines through polar coordinate transformation, and fitting the reference point center in combination with the least squares method;

[0071] The chip positioning unit includes: quickly locating the chip area under low resolution, refining the angle matching under high resolution, extracting chip pin features through Harris corner detection based on the point set registration algorithm, building the transformation matrix between the ideal model and the actual point set, and combining rotation, translation and scaling parameters to calculate the chip's position matrix relative to the machine coordinate system.

[0072] The PCB positioning unit specifically includes:

[0073] Dynamically divide the grayscale interval and linearly stretch the reference point area, for example, mapping the grayscale value to [0,255], compressing the background grayscale interval, for example, setting the reference point grayscale range to [80,150], and the background to [0,79] and [151,255]. Optimize the edge reference points through dual thresholds. The optimization process includes:

[0074] A grayscale threshold interval is set. Edge grayscale points above the grayscale threshold interval are considered strong edges, and edge grayscale points below the grayscale threshold interval are considered weak edges. Only weak edges connected to strong edges are retained to avoid interference from isolated noise points. The contours of all reference points are extracted, and parabolic fitting is performed on the discrete edge points detected by Canny. The sub-pixel edge position is calculated to obtain a sub-pixel edge image, reducing the reference point center positioning error from ±1 pixel to ±0.1 pixel.

[0075] The sub-pixel edge image is projected into the polar coordinate system, and the grayscale integral projection at each angle in the polar coordinate system is calculated. The cross-edge lines are detected and deleted through Hough transform. The remaining candidate lines after deleting the cross-edge lines are grayscale fitted, and the calculated intersection of the grayscale fitting is used as the reference point center, which has stronger anti-interference ability.

[0076] The chip positioning unit specifically includes:

[0077] A multi-resolution positioning strategy is used to quickly screen candidate chip areas in the global image. When the resolution is low, the image is downsampled and morphological operations are used to extract the approximate chip outline, including dilation and erosion.

[0078] When in high-resolution condition, the approximate chip outline is refined into a candidate chip area, and local feature extraction is performed. The local feature extraction includes: extracting chip pin feature points in the candidate area using Harris corner detection.

[0079] Extracting chip pin feature points specifically includes:

[0080] Calculate the gradient of each reference pixel point in the refined candidate chip area to describe the local changes in the image. Construct the gradient covariance matrix of each reference pixel point based on the gradient information. The covariance matrix captures the structural characteristics of the local image. The mathematical expression of the covariance matrix is:

[0081] ;

[0082] in, and are the gradients of the image in the x and y directions, respectively, where x and y are the reference pixels in the image;

[0083] Calculate the Harris corner response value. The mathematical expression of the Harris corner response value is:

[0084] ;

[0085] Wherein, k is an empirical constant, and its value is [0.04, 0.06];

[0086] Filter the reference pixel points whose corner response values ​​are higher than the preset response threshold, retain the gradient maximum point, and perform eigenvalue decomposition based on the basic matrix calculation on the covariance matrix to obtain the eigenvalue and , the eigenvalue indicates the direction and degree of image change at that point, constructs an ideal chip model, and outputs the chip pin feature points.

[0087] Building an ideal chip model includes:

[0088] Perform ICP iterative closest point matching based on the chip pin feature points and the pre-extracted actual point set, calculate the rotation matrix, translation vector and scaling factor, and generate the pose matrix;

[0089] The mathematical expression of the pose matrix is:

[0090] ;

[0091] Among them, N is the pose matrix, R is the rotation matrix, T is the translation vector, and S is the scaling factor;

[0092] The local coordinate system of the chip is converted to the machine coordinate system to obtain the actual position and orientation of the chip. The reference point coordinates in the machine coordinate system, the rotation angle of the chip relative to the machine coordinate system, and the chip pose matrix are output. The reference point coordinates are the chip pin feature points.

[0093] The decision module includes a planning unit and a control unit:

[0094] Planning units include:

[0095] Generate a multi-dimensional collaborative operation path based on the chip pin feature points, perform non-repetitive coding on the chip placement machine equipment and obtain the operation code, and build the mechanical equipment connecting rod coordinate system based on the operation code;

[0096] Generating a multi-dimensional collaborative operation path based on the chip pin feature points specifically includes:

[0097] Chip pin feature points are randomly extracted as path nodes, and a collision-free path is generated using the RRT algorithm. Multi-objective optimization of the chip placement operation path is performed considering chip size constraints. The optimization objectives include path length, time, and safety.

[0098] Obtaining the job code includes:

[0099] Generate time series path points based on the placement machine's beat cycle. Superimpose the Z-axis height coordinate information of the machine coordinate system of the chip pin feature points on the path of the discrete time series path points to generate a three-dimensional trajectory path. According to the process stage, the corresponding code prefix is ​​assigned to each process stage. The process stage includes loading, placement and inspection, and the three-dimensional dynamic operation code is output.

[0100] Constructing the mechanical equipment link coordinate system includes:

[0101] An adaptive mechanical device coordinate system is constructed through forward kinematics and inverse kinematics to control the position and posture of the end effector of the mechanical device. The position and posture of the end effector are calibrated and annotated in the mechanical device coordinate system. The position and posture of the end effector of the mechanical device are obtained by chain multiplication of the link transformation matrix. The joint angles of the end effector of the mechanical device are optimized through the particle swarm algorithm, and precise control is achieved in combination with the Jacobian matrix constraint.

[0102] The link transformation matrix expresses the position and posture of each link relative to the previous link through matrix multiplication. These transformation matrices are gradually accumulated and the position and posture of the end effector are obtained through chain multiplication.

[0103] The Jacobian matrix is ​​used to describe the relationship between the movement of the end effector in the joint space and the workspace. Through the Jacobian matrix, the relationship between the velocity, acceleration and joint angular velocity of the end effector can be analyzed, and then precise control can be performed. When performing inverse kinematics calculations, by combining the Jacobian matrix constraints, it can be ensured that the solution of the joint angle meets physical limitations, such as the joint's range of motion, speed limit, etc., thereby achieving precise control.

[0104] The control unit is used to perform master-slave collaborative formation control on the multi-dimensional collaborative operation path:

[0105] Master-slave collaborative formation control includes global path planning of the master device and tracking feedback of the slave devices;

[0106] The master device's global path planning includes: inputting the chip placement task sequence and the global map, using the improved APF-RRT algorithm for path planning, constructing the gravitational field of the target point and the repulsive field of obstacles, and generating a collision-free and smooth optimized path;

[0107] Path planning using the improved APF-RRT algorithm includes:

[0108] S01: Initialize the optimization environment and set the global path parameters. The global path parameters include the initial joint coordinates, angles, and obstacle information of the end effector of the robot. The obstacle information includes the coordinate set of the robot link coordinate system that the end effector of the robot is prohibited from reaching. Set the optimization target to the expected joint coordinates.

[0109] S02: Initialize global path planning based on the multi-dimensional collaborative operation path;

[0110] S03: Calculate the resultant force on the end effector of the current mechanical device. The resultant force is the resultant force of all forces obtained by force analysis based on Newton's laws of mechanics. Set a resultant force threshold F_min. When the resultant force is greater than the resultant force threshold F_min, it indicates that the current path planning is valid. Continue to detect the resultant force in real time and execute step S05.

[0111] When the resultant force is less than the resultant force threshold F_min, it indicates that the current path planning is invalid, and step S04 is executed;

[0112] S04: Perform a limited number of searches using the improved RRT algorithm. These searches attempt to search through different paths and select the node closest to the current joint angle as the temporary target joint angle, thereby providing a new path planning direction. Select the node closest to the current key angle on the RRT growth tree as the temporary target joint angle, and continue to execute step S02.

[0113] The improved RRT algorithm includes improving the node selection strategy and introducing a more efficient sampling method or pruning strategy to improve the path planning efficiency. The improved RRT algorithm is an existing technology.

[0114] S05: Determine whether the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point: when the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point, the path planning optimization task is completed and the loop is exited;

[0115] When the joint angle of the current end effector of the mechanical device does not reach the expected coordinate of the target joint point, return to step S02.

[0116] Feedback from device tracking includes:

[0117] Utilizing a feedback linearization control architecture and error compensation based on the Jacobian matrix, the LQR controller is used to establish a six-dimensional state equation and solve the optimal control law to achieve precise trajectory tracking. A virtual spring-damper model is used to maintain the desired spacing between master and slave devices. A distributed consensus algorithm is used to synchronize device states in the communication network to ensure the collaborative operation of master and slave devices.

[0118] The execution modules include:

[0119] Based on PID control theory, a three-loop controller of position, velocity and acceleration is designed. An error model is constructed according to the multi-dimensional collaborative operation path output by the decision module. The S-shaped curve optimizes the acceleration and deceleration process of the motion trajectory to reduce mechanical vibration in the chip placement environment. The chip position deviation is continuously monitored through the visual system, and the fuzzy adaptive control is used to adjust the placement head posture. The online calibration algorithm is used to compensate for the visual system drift caused by environmental changes.

[0120] Building an error model involves:

[0121] Define the errors of position, velocity and acceleration, and obtain the real-time actual position of the chip based on the mechanical equipment connecting rod coordinate system output by the decision module. Compare the coordinates with the preset target position to obtain the error vector.

[0122] S-curve acceleration and deceleration optimization includes:

[0123] Acceleration planning generates acceleration curves through quintic polynomial interpolation method, constraining the initial / final velocity, acceleration and chip displacement continuity.

[0124] The PCB reference points and chip pin feature points are updated according to the preset cycle, the offset of the mechanical equipment link coordinate system is calculated, the calibration error model is fitted using the RANSAC algorithm, and the impact of the equipment operation time and temperature on the visual system is calculated in real time. The calibration error is fed back as an error function to the multi-dimensional collaborative operation path, and the position and posture of the end effector are corrected through master-slave collaborative formation control to maintain an accurate operation path.

[0125] The present invention adopts improved Radon transform and polar coordinate screening, combined with parabola fitting to achieve sub-pixel level reference point center positioning, which improves the accuracy by 10 times compared with traditional methods. Through dual threshold segmentation and background compression, it reduces noise interference and enhances the robustness of feature extraction. Multi-resolution chip positioning enables precise positioning of the chip in the machine coordinate system. The improved APF-RRT algorithm combines the gravitational-repulsive field of the artificial potential field with RRT random search, dynamically plans collision-free paths, optimizes path length, time, and safety, and uses master-slave collaborative control to maintain collaborative spacing between devices. Three-dimensional dynamic operation coding generates a three-dimensional trajectory path with Z-axis height to adapt to flexible production needs. Quintic polynomial interpolation is used to generate a smooth acceleration curve, reduce mechanical vibration, and improve patch stability. The calibration error is fitted through the RANSAC algorithm, and the influence of environmental factors such as temperature and mechanical drift are compensated in real time. Visual positioning (perception) → path planning (decision-making) → dynamic control (execution) forms a closed loop to achieve adaptive adjustment of flexible production. Combining technologies from multiple fields such as machine vision, robotics, optimization algorithms, and control theory, it breaks through the limitations of a single method, is suitable for micron-level patch needs, supports multi-variety and small-batch production, and adapts to the trend of intelligent manufacturing. The dynamic error compensation mechanism copes with environmental interference and improves the yield rate. Through three major innovations: vision-enhanced positioning, intelligent collaborative control, and dynamic error compensation, the system achieves closed-loop management of chip placement production with higher precision, faster response, and greater adaptability than traditional methods.

[0126] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. Chip placement collaborative control system for flexible production, characterized by: Including perception module, decision module and execution module: The perception module is used to collect PCB and chip images through an industrial camera and locate the chip reference point position through machine vision; The decision module is used for path planning and coordinated control; The execution module is used for multi-dimensional collaborative operations; The sensing module includes a PCB positioning unit and a chip positioning unit: The PCB positioning unit includes locating a cross-shaped reference point on the PCB board and establishing a global coordinate system. The establishment of the global coordinate system specifically includes: using piecewise linear transformation to enhance the contrast of the image collected during the chip placement process, using Gaussian filtering to suppress image noise and retain edge details, extracting the reference point contour through Canny edge detection, improving Radon transformation to detect the straight line feature of the cross-shaped reference point, screening candidate lines through polar coordinate transformation, and fitting the reference point center in combination with the least squares method; The chip positioning unit includes: quickly locating the chip area under low resolution conditions, refining the angle matching under high resolution conditions, extracting chip pin features through Harris corner detection based on a point set registration algorithm, constructing a transformation matrix between the ideal model and the actual point set, and combining rotation, translation and scaling parameters to calculate the chip's pose matrix relative to the machine coordinate system; The chip positioning unit specifically includes: A multi-resolution positioning strategy is used to quickly screen candidate chip areas in the global image. When the resolution is low, the image is downsampled and morphological operations are used to extract the approximate chip outline, including dilation and erosion. When in high-resolution condition, the approximate chip outline is refined into candidate chip areas and local features are extracted. Local feature extraction includes: extracting chip pin feature points in the candidate area using Harris corner detection; Extracting chip pin feature points specifically includes: Calculate the gradient of each benchmark pixel point in the refined candidate chip area, and construct the gradient covariance matrix of each benchmark pixel point based on the gradient information. The mathematical expression of the covariance matrix is: ； Where Ix and Iy are the gradients of the image in the x and y directions respectively, and x and y are the reference pixels in the image; Calculate the Harris corner response value, the mathematical expression of the Harris corner response value is: ； Wherein, k is an empirical constant, and its value is [0.04, 0.06]; Filter the reference pixel points whose corner response values ​​are higher than the preset response threshold, retain the gradient maximum point, and perform eigenvalue decomposition based on the basic matrix calculation on the covariance matrix to obtain the eigenvalue and , build an ideal chip model and output the chip pin feature points; Building an ideal chip model includes: Perform ICP iterative closest point matching based on the chip pin feature points and the pre-extracted actual point set, calculate the rotation matrix, translation vector and scaling factor, and generate a pose matrix; The mathematical expression of the pose matrix is: ； Among them, N is the pose matrix, R is the rotation matrix, T is the translation vector, and S is the scaling factor; The chip's local coordinate system is converted to the machine coordinate system to obtain the actual position and orientation of the chip. The reference point coordinates in the machine coordinate system, the rotation angle of the chip relative to the machine coordinate system, and the chip pose matrix are output. The reference point coordinates are the chip pin feature points.

2. The chip placement collaborative control system for flexible production according to claim 1, characterized in that: The PCB positioning unit specifically includes: Dynamically divide the grayscale interval, linearly stretch the reference point area, compress the background grayscale interval, and optimize the edge reference points through dual thresholds. The optimization process includes: Set the grayscale threshold interval. Edge grayscale points above the grayscale threshold interval are considered strong edges, and edge grayscale points below the grayscale threshold interval are considered weak edges. Only weak edges connected to strong edges are retained. The contours of all reference points are extracted. Parabolic fitting is performed on the discrete edge points detected by Canny. The sub-pixel edge position is calculated to obtain a sub-pixel edge image. The sub-pixel edge image is projected into a polar coordinate system, and the grayscale integral projection at each angle in the polar coordinate system is calculated. The cross-edge lines are detected and deleted by Hough transform, and the remaining candidate lines after deleting the cross-edge lines are grayscale fitted, and the calculated intersection of the grayscale fitting is used as the reference point center.

3. The chip placement collaborative control system for flexible production according to claim 1, characterized in that: The decision module includes a planning unit and a control unit: The planning unit includes: Generate a multi-dimensional collaborative operation path based on the chip pin feature points, perform non-repetitive coding on the chip placement machine equipment and obtain the operation code, and build the mechanical equipment connecting rod coordinate system based on the operation code; Generating a multi-dimensional collaborative operation path based on the chip pin feature points specifically includes: The chip pin feature points are randomly extracted as path nodes, and a collision-free path is generated using the RRT algorithm. The chip placement operation path is optimized for multiple objectives, including path length, time, and safety. Obtaining the job code includes: Based on the placement machine's beat cycle, a time series path point is generated. The Z-axis height coordinate information of the machine coordinate system of the chip pin feature point is superimposed on the path of the discrete time series path point to generate a three-dimensional trajectory path. The coding prefix corresponding to each process stage is assigned according to the process stage, which includes loading, placement and inspection, and the three-dimensional dynamic operation code is output; Constructing the mechanical equipment link coordinate system includes: An adaptive mechanical device coordinate system is constructed through forward kinematics and inverse kinematics to control the position and posture of the end effector of the mechanical device. The position and posture of the end effector are calibrated and annotated in the mechanical device coordinate system. The position and posture of the end effector of the mechanical device are obtained by chain multiplication of the link transformation matrix. The joint angles of the end effector of the mechanical device are optimized through the particle swarm algorithm, and precise control is achieved in combination with the Jacobian matrix constraint.

4. The chip placement collaborative control system for flexible production according to claim 3, characterized in that: The control unit is used to perform master-slave collaborative formation control on the multi-dimensional collaborative operation path: Master-slave collaborative formation control includes global path planning of the master device and tracking feedback of the slave devices; The master device's global path planning includes: inputting the chip placement task sequence and the global map, using the improved APF-RRT algorithm for path planning, constructing the gravitational field of the target point and the repulsive field of obstacles, and generating a collision-free and smooth optimized path; Path planning using the improved APF-RRT algorithm includes: S01: Initialize the optimization environment and set global path parameters, which include the initial joint coordinates, angles, and obstacle information of the end effector of the mechanical device. The obstacle information includes the coordinate set of the mechanical device link coordinate system that the end effector of the mechanical device is prohibited from reaching. Set the optimization target to the expected joint coordinates. S02: Initialize global path planning based on the multi-dimensional collaborative operation path; S03: Calculate the resultant force on the end effector of the current mechanical device. The resultant force is the resultant force of all forces obtained by force analysis according to Newton's law of mechanics. Set a resultant force threshold F_min. When the resultant force is greater than the resultant force threshold F_min, it indicates that the current path planning is valid. Continue to detect the resultant force in real time and execute step S05. When the resultant force is less than the resultant force threshold F_min, it indicates that the current path planning is invalid, and step S04 is executed; S04: Perform a limited search using the improved RRT algorithm, select the node on the RRT growth tree closest to the current key angle as the temporary target joint angle, and continue to execute step S02; S05: Determine whether the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point: when the joint angle of the current end effector of the mechanical device has reached the expected coordinate of the target joint point, the path planning optimization task is completed and the loop is exited; When the joint angle of the current mechanical device end effector does not reach the expected coordinate of the target joint point, return to step S02; Feedback from device tracking includes: A feedback linearization control architecture is used to perform error compensation based on the Jacobian matrix. The six-dimensional state equation is established through the LQR controller and the optimal control law is solved to achieve accurate trajectory tracking. A virtual spring-damper model is used to maintain the desired spacing between master and slave devices. A distributed consensus algorithm is used to synchronize device states in the communication network.

5. The chip placement collaborative control system for flexible production according to claim 4, characterized in that: The execution module includes: Based on PID control theory, a three-loop controller for position, velocity, and acceleration is designed. An error model is constructed based on the multi-dimensional collaborative operation path output by the decision module. An S-shaped curve is used to optimize the acceleration and deceleration process of the motion trajectory. The chip position deviation is continuously monitored through the vision system, and the placement head posture is adjusted using fuzzy adaptive control. An online calibration algorithm is used to compensate for vision system drift caused by environmental changes. Building an error model involves: Define the errors of position, velocity and acceleration, and obtain the real-time actual position of the chip based on the mechanical equipment connecting rod coordinate system output by the decision module. Compare the coordinates with the preset target position to obtain the error vector.

6. The chip placement collaborative control system for flexible production according to claim 5, characterized in that: S-curve acceleration and deceleration optimization includes: Acceleration planning generates acceleration curves through quintic polynomial interpolation, constraining the initial / final velocity, acceleration, and chip displacement continuity; The PCB reference points and chip pin feature points are updated according to the preset cycle, the offset of the mechanical equipment link coordinate system is calculated, the calibration error model is fitted using the RANSAC algorithm, and the impact of the equipment operation time and temperature on the visual system is calculated in real time. The calibration error is fed back as an error function to the multi-dimensional collaborative operation path, and the position and posture of the end effector are corrected through master-slave collaborative formation control.

Images