Vision-assisted chip patch positioning method and system

Abstract

The invention discloses a vision-assisted chip mounting positioning method and system, and relates to the field of chip mounters, and the method comprises the steps: obtaining a current image of a to-be-mounted circuit board, and obtaining a target coordinate set of a to-be-mounted element in combination with a circuit board design drawing; marking the current coordinate of the suction nozzle of the chip mounter as a starting point coordinate; obtaining an initial coordinate set of the to-be-mounted component according to the current image of the to-be-mounted component; and according to the initial coordinate set and the target coordinate set, calculating coordinates of a midpoint and an end point of movement of the suction nozzle of the chip mounter. And based on the initial coordinate set and the target coordinate set, carrying out rotation adjustment on the angle of the suction nozzle by using an angle adjustment model, so that the to-be-mounted component can accurately rotate to the angle consistent with the target position, the component angle is prevented from deviating, and based on the starting point, the middle point and the end point, planning a suction nozzle path by using a path planning model. Therefore, the mechanical arm drives the suction nozzle to transport the to-be-mounted component along the optimal path, and the position deviation of the component is prevented. According to the invention, rapid and accurate positioning of the chip mounter is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of chip placement machines, and in particular to a vision-assisted chip placement positioning method and system. Background Art

[0002] Chip mounter is a very critical equipment in the modern electronic manufacturing industry. It is used to accurately mount surface mount components (such as chips, diodes, capacitors, resistors, etc.) onto circuit boards. In recent years, with the increasing miniaturization, complexity of functions and improvement of production requirements of electronic products, the technology of chip mounter has also been continuously developed.

[0003] Most of the existing positioning methods of placement machines detect whether the placement of the patch components is correct by comparing the actual position with the theoretical position, so as to determine whether to correct it. However, the correction must be made after the component is placed, and direct and accurate positioning cannot be achieved, which reduces the efficiency of the placement to a certain extent. Summary of the invention

[0004] The technical problem solved by the present invention is that most of the existing technologies perform deviation correction after placing the components, which cannot achieve direct and accurate positioning, thus reducing the efficiency of patching to a certain extent.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a visually assisted chip placement positioning method, comprising:

[0006] Step S100, constructing a spatial coordinate system and determining the origin position;

[0007] Step S200, obtaining a current image of the circuit board to be mounted, analyzing the current position of the circuit board to be mounted in the spatial coordinate system, obtaining a circuit board design drawing and establishing a circuit board coordinate system, constructing a standard three-dimensional model drawing according to the circuit board design drawing, and calculating a corner point coordinate set of a component in the standard three-dimensional model drawing in the circuit board coordinate system, converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set of the component in the spatial coordinate system by means of coordinate conversion, and recording the corner point coordinate set of the component in the spatial coordinate system as a target coordinate set;

[0008] Step S300: collecting the current image of the nozzle of the placement machine, obtaining the current coordinates of the nozzle of the placement machine, recording them as the starting coordinates, collecting the current image of the component to be mounted, analyzing the current state of the component to be mounted, obtaining the current corner point coordinate set of the component to be mounted, recording them as the initial coordinate set;

[0009] Step S400: Calculate the midpoint coordinates and the end point coordinates according to the initial coordinate set and the target coordinate set, input the initial coordinate set, the target coordinate set, the starting point coordinates, the midpoint coordinates and the end point coordinates into the pre-trained path planning model and the angle adjustment model respectively, perform optimal path planning for the moving trajectory of the placement machine nozzle, and adjust the angle of the placement machine nozzle.

[0010] Preferably, the step S1 specifically includes:

[0011] Step S101: taking the center of the placement machine workbench as the coordinate origin of the three-dimensional space coordinate system;

[0012] Step S102: The left and right directions of the working direction of the placement machine are taken as the X-axis of the three-dimensional space coordinate system, and the right is taken as the positive direction of the X-axis;

[0013] Step S103: The front and rear directions of the working direction of the placement machine are taken as the Y axis of the three-dimensional space coordinate system, and the front direction is the positive direction of the Y axis;

[0014] Step S104: The up and down directions of the working direction of the placement machine are used as the Z axis of the three-dimensional space coordinate system, and the positive direction of the Z axis is above;

[0015] Step S105: The rotation axis of the suction nozzle is set as the θ axis.

[0016] Preferably, the step S200 specifically includes:

[0017] Step S201: using a binocular camera to collect a current image of the circuit board to be mounted, graying and denoising the current image of the circuit board to be mounted to obtain a first processed image, and using a target detection algorithm to detect the position of the circuit board to be mounted;

[0018] Step S202: Obtain a circuit board design drawing, and analyze the position, direction, and connection relationship between components in the circuit board design drawing;

[0019] Step S203: establishing a circuit board coordinate system, constructing the positions, directions and connection relationships between components in the design drawing into a standard three-dimensional model diagram, and calculating the corner point coordinate set of the components in the circuit board coordinate system according to the positions of the standard three-dimensional model diagram in the circuit board coordinate system;

[0020] Step S204: converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set in the space coordinate system by coordinate translation, rotation and scaling, that is, the target coordinate set;

[0021] Step S205: number each to-be-mounted component one by one, and each to-be-mounted component corresponding to a number has its corresponding target coordinate set.

[0022] Preferably, in step S204, the coordinate conversion formula is:

[0023] x1=xT;

[0024] T=R z *T x *T s ;

[0025] Among them, x1 represents the coordinates in the space coordinate system, x represents the coordinates in the circuit board coordinate system, T represents the transformation matrix, and R z represents the rotation matrix, T x represents the translation matrix, T s Represents the scaling matrix.

[0026] Preferably, the step S300 specifically includes:

[0027] Step S301: acquiring a current image of a component to be mounted, performing grayscale processing and denoising processing on the current image of the component to be mounted to obtain a second processed image, identifying contour features of the second processed image, determining the shape, size and name of the component to be mounted, matching the shape, size and name with the number, and finding a target coordinate set of the component to be mounted;

[0028] Step S302: performing corner point analysis on the contour features of the second processed image, determining the current coordinates of each corner point in the spatial coordinate system, merging the current coordinates of each corner point and outputting them as a corner point coordinate set, recorded as an initial coordinate set;

[0029] Step S303: Acquire the current image of the nozzle of the placement machine, and determine the current coordinates of the nozzle of the placement machine according to the relative position of the nozzle and the robot arm.

[0030] Preferably, the training process of the path planning model is:

[0031] Collect multiple groups of first sample sets, and divide the first sample sets into a first training set and a first test set according to a preset ratio, wherein the first sample sets include a starting point coordinate A (x1, y1, z1), a midpoint coordinate B (x2, y2, z2), and an end point coordinate C (x3, y3, z3);

[0032] Taking the coordinates of the starting point, midpoint and end point as input and the shortest path as output for training, a preliminary path planning model is obtained;

[0033] The preliminary path planning model is tested using a first test set, and training is stopped when a first accuracy is reached to obtain a trained path planning model.

[0034] Preferably, the training process of the angle adjustment model is:

[0035] Collecting multiple groups of second sample sets, and dividing the second sample sets into a second training set and a second test set according to a preset ratio;

[0036] The second sample set includes an initial coordinate set M{M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)} and the target coordinate set N{N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)};

[0037] Taking the initial coordinate set M and the target coordinate set N as input and the rotation angle of the nozzle rotation axis as output for training, a preliminary angle adjustment model is obtained;

[0038] The preliminary angle adjustment model is trained using a second test set, and when a second accuracy is reached, training is stopped to obtain a trained angle adjustment model.

[0039] Preferably, step S400 includes:

[0040] Step S401: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)};

[0041] Step S402: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)};

[0042] Step S403: inputting the initial coordinate set M of the component to be mounted and the target coordinate set N of the component to be mounted into the trained angle adjustment model, and outputting the adjustment angle θ0;

[0043] Step S404: the nozzle rotates around the θ axis, and the rotation angle is θ0.

[0044] Preferably, step S400 further includes:

[0045] Step S405: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)}, calculate the center coordinates of the initial coordinate set, recorded as the midpoint (M x , M y , M z ), the midpoint is calculated as follows:

[0046] M x=(x1+x2+…+xm) / m;

[0047] M y =(y1+y2+…+ym) / m;

[0048] M z =(z1+z2+…+zm) / m;

[0049] Among them, M x is the x coordinate of the midpoint, M y is the y coordinate of the midpoint, M z is the z coordinate of the midpoint;

[0050] Step S406: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)}, calculate the center coordinates of the target coordinate set using the same logic as in step S405 as the end point (N x , N y , N z );

[0051] Step S407: Obtain the current coordinates of the nozzle as the starting point A (x1, y1, z1);

[0052] Step S408: The starting point A (x1, y1, z1), the midpoint (M x , M y , M z )、End point(N x , N y , N z ) is input into the trained path planning model and the optimal path is output;

[0053] Step S409: The robot arm drives the nozzle of the placement machine to move along the optimal path.

[0054] On the other hand, the present invention provides a vision-assisted chip placement positioning system, including a model building module, an image acquisition module, an image processing module, a solution generation module and a control module;

[0055] The model building module is used to build a path planning model and an angle adjustment model;

[0056] The image acquisition module is used to acquire the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter;

[0057] The image processing module is used to perform image processing and image analysis on the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter to obtain an initial coordinate set, a target coordinate set, a starting point, a midpoint, and an end point;

[0058] The solution generation module is used to generate a positioning solution based on the initial coordinate set, the target coordinate set, the starting point, the midpoint and the end point, using a path planning model and an angle adjustment model, to perform optimal path planning on the moving path of the placement machine nozzle, and to adjust the angle of the placement machine nozzle;

[0059] The control module is used to receive the positioning solution output by the path planning model and the angle adjustment model, and control the movement of the robot arm and the rotation of the rotation axis according to the positioning solution.

[0060] The beneficial effects of the present invention are as follows: by collecting the current image of the circuit board to be mounted, the current image of the component to be mounted and the current image of the nozzle of the placement machine, an initial coordinate set, a target coordinate set, a starting point, a midpoint and an end point are obtained; based on the initial coordinate set and the target coordinate set, the angle of the nozzle is adjusted by using an angle adjustment model, so that the component to be mounted can be successfully rotated to an angle consistent with the target position to prevent the angle of the component from shifting; based on the starting point, the midpoint and the end point, the path planning model is used to plan the path of the nozzle, so that the robot arm drives the nozzle to transport the component to be mounted along the optimal path to prevent the position of the component from shifting, thereby achieving accurate and rapid positioning of the chip placement. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the basic flow of a visually assisted chip placement positioning method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0063] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a visually assisted chip placement positioning method, comprising:

[0064] Step S100, constructing a spatial coordinate system and determining the origin position;

[0065] Step S200, obtaining a current image of the circuit board to be mounted, analyzing the current position of the circuit board to be mounted in the spatial coordinate system, obtaining a circuit board design drawing and establishing a circuit board coordinate system, constructing a standard three-dimensional model drawing according to the circuit board design drawing, and calculating a corner point coordinate set of a component in the standard three-dimensional model drawing in the circuit board coordinate system, converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set of the component in the spatial coordinate system by means of coordinate conversion, and recording the corner point coordinate set of the component in the spatial coordinate system as a target coordinate set;

[0066] Step S300: collecting the current image of the nozzle of the placement machine, obtaining the current coordinates of the nozzle of the placement machine, recording them as the starting coordinates, collecting the current image of the component to be mounted, analyzing the current state of the component to be mounted, obtaining the current corner point coordinate set of the component to be mounted, recording them as the initial coordinate set;

[0067] Step S400: Calculate the midpoint coordinates and the end point coordinates according to the initial coordinate set and the target coordinate set, input the initial coordinate set, the target coordinate set, the starting point coordinates, the midpoint coordinates and the end point coordinates into the pre-trained path planning model and the angle adjustment model respectively, perform optimal path planning for the moving trajectory of the placement machine nozzle, and adjust the angle of the placement machine nozzle.

[0068] Specifically, by collecting the current image of the circuit board to be mounted, the current image of the component to be mounted and the current image of the nozzle of the placement machine, the initial coordinate set, the target coordinate set, the starting point, the midpoint and the end point are obtained. Based on the initial coordinate set and the target coordinate set, the angle of the nozzle is adjusted by using the angle adjustment model, so that the component to be mounted can be successfully rotated to an angle consistent with the target position to prevent the angle of the component from shifting. Based on the starting point, the midpoint and the end point, the path planning model is used to plan the path of the nozzle, so that the robot arm drives the nozzle to transport the component to be mounted along the optimal path to prevent the position of the component from shifting, thereby achieving precise positioning of the chip placement.

[0069] The step S1 specifically includes:

[0070] Step S101: taking the center of the placement machine workbench as the coordinate origin of the three-dimensional space coordinate system;

[0071] Step S102: The left and right directions of the working direction of the placement machine are taken as the X-axis of the three-dimensional space coordinate system, and the right is taken as the positive direction of the X-axis;

[0072] Step S103: The front and rear directions of the working direction of the placement machine are taken as the Y axis of the three-dimensional space coordinate system, and the front direction is the positive direction of the Y axis;

[0073] Step S104: The up and down directions of the working direction of the placement machine are used as the Z axis of the three-dimensional space coordinate system, and the positive direction of the Z axis is above;

[0074] Step S105: The rotation axis of the suction nozzle is set as the θ axis.

[0075] Specifically, the spatial coordinate system helps the placement machine define the position of each component relative to the circuit board, ensuring that each component is placed in the correct position and direction. Through the coordinate system, the placement machine can accurately calculate the angle, direction and position of each component to avoid component orientation errors or offsets.

[0076] The step S200 specifically includes:

[0077] Step S201: using a binocular camera to collect a current image of the circuit board to be mounted, graying and denoising the current image of the circuit board to be mounted to obtain a first processed image, and using a target detection algorithm to detect the position of the circuit board to be mounted;

[0078] Step S202: Obtain a circuit board design drawing, and analyze the position, direction, and connection relationship between components in the circuit board design drawing;

[0079] Step S203: establishing a circuit board coordinate system, constructing the positions, directions and connection relationships between components in the design drawing into a standard three-dimensional model diagram, and calculating the corner point coordinate set of the components in the circuit board coordinate system according to the positions of the standard three-dimensional model diagram in the circuit board coordinate system;

[0080] Step S204: converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set in the space coordinate system by coordinate translation, rotation and scaling, that is, the target coordinate set;

[0081] Step S205: number each to-be-mounted component one by one, and each to-be-mounted component corresponding to a number has its corresponding target coordinate set.

[0082] Specifically, in this embodiment, the target detection algorithm may be edge detection, Hough transform in traditional computer vision methods, or a deep learning method, such as the YOLO target detection model. Assuming that there is a rectangular component in the design drawing, whose size is 10mm×5mm, the standard 3D model of the component may be a rectangular block with a length of 10mm and a width of 5mm.

[0083] In step S204, the coordinate conversion formula is:

[0084] x1=xT;

[0085] T=R z *T x *T s ;

[0086] Among them, x1 represents the coordinates in the space coordinate system, x represents the coordinates in the circuit board coordinate system, T represents the transformation matrix, and R z represents the rotation matrix, T x represents the translation matrix, T s Represents the scaling matrix.

[0087] The step S300 specifically includes:

[0088] Step S301: acquiring a current image of a component to be mounted, performing grayscale processing and denoising processing on the current image of the component to be mounted to obtain a second processed image, identifying contour features of the second processed image, determining the shape, size and name of the component to be mounted, matching the shape, size and name with the number, and finding a target coordinate set of the component to be mounted;

[0089] Step S302: performing corner point analysis on the contour features of the second processed image, determining the current coordinates of each corner point in the spatial coordinate system, merging the current coordinates of each corner point and outputting them as a corner point coordinate set, recorded as an initial coordinate set;

[0090] Step S303: Acquire the current image of the nozzle of the placement machine, and determine the current coordinates of the nozzle of the placement machine according to the relative position of the nozzle and the robot arm.

[0091] Specifically, the patch component is identified to obtain its shape, size and name. For example, if a rectangular component with a size of 10mm×5mm is detected, the matched component name may be "SOT-23 transistor". The number corresponding to the SOT-23 transistor is found, and the target coordinate set of the component is determined, thereby determining the position and angle at which the component should be placed, and then calculating the target coordinate point set of the component corner points.

[0092] The training process of the path planning model is:

[0093] Collect multiple groups of first sample sets, and divide the first sample sets into a first training set and a first test set according to a preset ratio, wherein the first sample sets include a starting point coordinate A (x1, y1, z1), a midpoint coordinate B (x2, y2, z2), and an end point coordinate C (x3, y3, z3);

[0094] Taking the coordinates of the starting point, midpoint and end point as input and the shortest path as output for training, a preliminary path planning model is obtained;

[0095] The preliminary path planning model is tested using a first test set, and training is stopped when a first accuracy is reached to obtain a trained path planning model.

[0096] Specifically, the training goal of the model is to predict the shortest path output by inputting the starting point, midpoint and end point coordinates, which is crucial for the path planning and navigation of the placement machine nozzle. After path planning, the shortest path can effectively improve the efficiency of the nozzle in transporting components to be mounted, saving time and resources while reducing unnecessary consumption.

[0097] The training process of the angle adjustment model is:

[0098] Collecting multiple groups of second sample sets, and dividing the second sample sets into a second training set and a second test set according to a preset ratio;

[0099] The second sample set includes an initial coordinate set M{M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)} and the target coordinate set N{N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)};

[0100] Taking the initial coordinate set M and the target coordinate set N as input and the rotation angle of the nozzle rotation axis as output for training, a preliminary angle adjustment model is obtained;

[0101] The preliminary angle adjustment model is trained using a second test set, and when a second accuracy is reached, training is stopped to obtain a trained angle adjustment model.

[0102] Specifically, the initial coordinate set M includes the current spatial position and posture of the component to be mounted, and the target coordinate set N represents the position and posture that the component to be mounted needs to reach, which is a target area and has a specific direction. The offset angle between the two coordinate sets is obtained through the initial coordinate set M and the target coordinate set N, thereby determining the rotation angle of the rotation axis, that is, the adjustment angle is calculated through the input coordinate information so that the suction nozzle can accurately drive the component to be mounted to rotate to prevent the component angle from shifting.

[0103] Step S400 includes:

[0104] Step S401: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)};

[0105] Step S402: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)};

[0106] Step S403: inputting the initial coordinate set M of the component to be mounted and the target coordinate set N of the component to be mounted into the trained angle adjustment model, and outputting the adjustment angle θ0;

[0107] Step S404: the nozzle rotates around the θ axis, and the rotation angle is θ0.

[0108] Specifically, through image acquisition and image processing, the coordinates of the corner points of the patch component are calculated to obtain the initial coordinate set M of the component to be patched, and the target coordinate set N is obtained. The two coordinate sets are input into the pre-built angle adjustment model, so that the suction nozzle can accurately drive the component to be patched to rotate and prevent the angle of the component from shifting.

[0109] Step S400 also includes:

[0110] Step S405: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)}, calculate the center coordinates of the initial coordinate set, recorded as the midpoint (M x , M y , M z ), the midpoint is calculated as follows:

[0111] M x =(x1+x2+…+xm) / m;

[0112] M y =(y1+y2+…+ym) / m;

[0113] M z =(z1+z2+…+zm) / m;

[0114] Among them, M x is the x coordinate of the midpoint, M y is the y coordinate of the midpoint, M z is the z coordinate of the midpoint;

[0115] Step S406: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)}, calculate the center coordinates of the target coordinate set using the same logic as in step S405 as the end point (N x , N y , N z );

[0116] Step S407: Obtain the current coordinates of the nozzle as the starting point A (x1, y1, z1);

[0117] Step S408: The starting point A (x1, y1, z1), the midpoint (M x , M y , M z )、End point(N x , N y , N z ) is input into the trained path planning model and the optimal path is output;

[0118] Step S409: The robot arm drives the nozzle of the placement machine to move along the optimal path.

[0119] Specifically, in this embodiment, the nozzle of the placement machine is regarded as a point, which simplifies the calculation steps. The center coordinates of the initial coordinate set of the component to be placed and the center coordinates of the target coordinate set are calculated as the midpoint and end point of the movement of the nozzle of the placement machine, respectively. The nozzle of the placement machine starts from the starting point and reaches the midpoint. After sucking the component to be placed, it moves according to the planned path and transports the component to be placed to the target position. The present invention adopts a machine learning model to improve the efficiency of the nozzle in conveying components, and locates the components in combination with the coordinate system, thereby improving the placement accuracy of the placement machine.

[0120] Embodiment 2 is a vision-assisted chip placement positioning system provided by the present invention, comprising a model building module, an image acquisition module, an image processing module, a solution generation module and a control module;

[0121] The model building module is used to build a path planning model and an angle adjustment model;

[0122] The image acquisition module is used to acquire the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter;

[0123] The image processing module is used to perform image processing and image analysis on the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter to obtain an initial coordinate set, a target coordinate set, a starting point, a midpoint, and an end point;

[0124] The solution generation module is used to generate a positioning solution based on the initial coordinate set, the target coordinate set, the starting point, the midpoint and the end point, using a path planning model and an angle adjustment model, to perform optimal path planning on the moving path of the placement machine nozzle, and to adjust the angle of the placement machine nozzle;

[0125] The control module is used to receive the positioning solution output by the path planning model and the angle adjustment model, and control the movement of the robot arm and the rotation of the rotation axis according to the positioning solution.

[0126] It should be understood by those skilled in the art that the embodiments of the present invention can be provided as methods, systems or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A vision-assisted chip placement positioning method, characterized in that: include: Step S100, constructing a spatial coordinate system and determining the origin position; Step S200, obtaining a current image of the circuit board to be mounted, analyzing the current position of the circuit board to be mounted in the spatial coordinate system, obtaining a circuit board design drawing and establishing a circuit board coordinate system, constructing a standard three-dimensional model drawing according to the circuit board design drawing, and calculating a corner point coordinate set of a component in the standard three-dimensional model drawing in the circuit board coordinate system, converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set of the component in the spatial coordinate system by means of coordinate conversion, and recording the corner point coordinate set of the component in the spatial coordinate system as a target coordinate set; Step S300: collecting the current image of the nozzle of the placement machine, obtaining the current coordinates of the nozzle of the placement machine, recording them as the starting coordinates, collecting the current image of the component to be mounted, analyzing the current state of the component to be mounted, obtaining the current corner point coordinate set of the component to be mounted, recording them as the initial coordinate set; Step S400: Calculate the midpoint coordinates and the end point coordinates according to the initial coordinate set and the target coordinate set, input the initial coordinate set, the target coordinate set, the starting point coordinates, the midpoint coordinates and the end point coordinates into the pre-trained path planning model and the angle adjustment model respectively, perform optimal path planning for the moving trajectory of the placement machine nozzle, and adjust the angle of the placement machine nozzle.

2. The visually assisted chip placement positioning method according to claim 1, characterized in that: The step S1 specifically includes: Step S101: taking the center of the placement machine workbench as the coordinate origin of the three-dimensional space coordinate system; Step S102: The left and right directions of the working direction of the placement machine are taken as the X-axis of the three-dimensional space coordinate system, and the right is taken as the positive direction of the X-axis; Step S103: The front and rear directions of the working direction of the placement machine are taken as the Y axis of the three-dimensional space coordinate system, and the front direction is the positive direction of the Y axis; Step S104: The up and down directions of the working direction of the placement machine are used as the Z axis of the three-dimensional space coordinate system, and the positive direction of the Z axis is above; Step S105: The rotation axis of the suction nozzle is set as the θ axis.

3. The visually assisted chip placement positioning method according to claim 1, characterized in that: The step S200 specifically includes: Step S201: using a binocular camera to collect a current image of the circuit board to be mounted, graying and denoising the current image of the circuit board to be mounted to obtain a first processed image, and using a target detection algorithm to detect the position of the circuit board to be mounted; Step S202: Obtain a circuit board design drawing, and analyze the position, direction, and connection relationship between components in the circuit board design drawing; Step S203: establishing a circuit board coordinate system, constructing the positions, directions and connection relationships between components in the design drawing into a standard three-dimensional model diagram, and calculating the corner point coordinate set of the components in the circuit board coordinate system according to the positions of the standard three-dimensional model diagram in the circuit board coordinate system; Step S204: converting the corner point coordinate set of the component in the circuit board coordinate system into the corner point coordinate set in the space coordinate system by coordinate translation, rotation and scaling, that is, the target coordinate set; Step S205: number each to-be-mounted component one by one, and each to-be-mounted component corresponding to a number has its corresponding target coordinate set.

4. The visually assisted chip placement positioning method according to claim 3, characterized in that: In step S204, the coordinate conversion formula is: x1=xT; T=R z *T x *T s ; Among them, x1 represents the coordinates in the space coordinate system, x represents the coordinates in the circuit board coordinate system, T represents the transformation matrix, and R z represents the rotation matrix, T x represents the translation matrix, T s Represents the scaling matrix.

5. The visually assisted chip placement positioning method according to claim 3, characterized in that: The step S300 specifically includes: Step S301: acquiring a current image of a component to be mounted, performing grayscale processing and denoising processing on the current image of the component to be mounted to obtain a second processed image, identifying contour features of the second processed image, determining the shape, size and name of the component to be mounted, matching the shape, size and name with the number, and finding a target coordinate set of the component to be mounted; Step S302: performing corner point analysis on the contour features of the second processed image, determining the current coordinates of each corner point in the spatial coordinate system, merging the current coordinates of each corner point and outputting them as a corner point coordinate set, recorded as an initial coordinate set; Step S303: Acquire the current image of the nozzle of the placement machine, and determine the current coordinates of the nozzle of the placement machine according to the relative position of the nozzle and the robot arm.

6. The visually assisted chip placement positioning method according to claim 1, characterized in that: The training process of the path planning model is: Collect multiple groups of first sample sets, and divide the first sample sets into a first training set and a first test set according to a preset ratio, wherein the first sample sets include a starting point coordinate A (x1, y1, z1), a midpoint coordinate B (x2, y2, z2), and an end point coordinate C (x3, y3, z3); Taking the coordinates of the starting point, midpoint and end point as input and the shortest path as output for training, a preliminary path planning model is obtained; The preliminary path planning model is tested using a first test set, and training is stopped when a first accuracy is reached to obtain a trained path planning model.

7. The visually assisted chip placement positioning method according to claim 5, characterized in that: The training process of the angle adjustment model is: Collecting multiple groups of second sample sets, and dividing the second sample sets into a second training set and a second test set according to a preset ratio; The second sample set includes an initial coordinate set M{M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)} and the target coordinate set N{N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)}; Taking the initial coordinate set M and the target coordinate set N as input and the rotation angle of the nozzle rotation axis as output for training, a preliminary angle adjustment model is obtained; The preliminary angle adjustment model is trained using a second test set, and when a second accuracy is reached, training is stopped to obtain a trained angle adjustment model.

8. The visually assisted chip placement positioning method according to claim 1, characterized in that: Step S400 includes: Step S401: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)}; Step S402: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)}; Step S403: inputting the initial coordinate set M of the component to be mounted and the target coordinate set N of the component to be mounted into the trained angle adjustment model, and outputting the adjustment angle θ0; Step S404: the nozzle rotates around the θ axis, and the rotation angle is θ0.

9. The visually assisted chip placement positioning method according to claim 1, characterized in that: Step S400 also includes: Step S405: Obtain the initial coordinate set M {M1(x1, y1, z1), M2(x2, y2, z2)...M m (xm, ym, zm)}, calculate the center coordinates of the initial coordinate set, recorded as the midpoint (M x , M y , M z ), the midpoint is calculated as follows: M x =(x1+x2+……+xm) / m; M y =(y1+y2+……+ym) / m; M z =(z1+z2+……+zm) / m; Among them, M x is the x coordinate of the midpoint, M y is the y coordinate of the midpoint, M z is the z coordinate of the midpoint; Step S406: Obtain the target coordinate set N {N1(x1, y1, z1), N2(x2, y2, z2)...N n (xn, yn, zn)}, calculate the center coordinates of the target coordinate set using the same logic as in step S405 as the end point (N x , N y , N z ); Step S407: Obtain the current coordinates of the nozzle as the starting point A (x1, y1, z1); Step S408: The starting point A (x1, y1, z1), the midpoint (M x , M y , N z )、End point(N x , N y , N z ) is input into the trained path planning model and the optimal path is output; Step S409: The robot arm drives the nozzle of the placement machine to move along the optimal path.

10. A vision-assisted chip placement positioning system, characterized in that: It includes a model building module, an image acquisition module, an image processing module, a solution generation module and a control module; The model building module is used to build a path planning model and an angle adjustment model; The image acquisition module is used to acquire the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter; The image processing module is used to perform image processing and image analysis on the current image of the circuit board to be mounted, the current image of the component to be mounted, and the current image of the nozzle of the mounter to obtain an initial coordinate set, a target coordinate set, a starting point, a midpoint, and an end point; The solution generation module is used to generate a positioning solution based on the initial coordinate set, the target coordinate set, the starting point, the midpoint and the end point, using a path planning model and an angle adjustment model, to perform optimal path planning on the moving path of the placement machine nozzle, and to adjust the angle of the placement machine nozzle; The control module is used to receive the positioning solution output by the path planning model and the angle adjustment model, and control the movement of the robot arm and the rotation of the rotation axis according to the positioning solution.

Images