Plug-in mounting method and device of electronic component, product, equipment and medium

Abstract

The embodiment of the invention provides an electronic component plug-in mounting method and device, a product, equipment and a medium, and the method comprises the steps: obtaining an image of an electronic component, determining a pin point coordinate of the electronic component according to the image of the electronic component, and determining a pin spacing value, a pin number and a target angle value according to the pin point coordinate, the target angle value is an angle value of a fitting straight line corresponding to the pin point coordinates, judging whether the electronic component is qualified or not according to the pin spacing value, the pin number and the target angle value, if the electronic component is qualified, obtaining a first circuit board image of the circuit board, and determining pin hole point coordinates of the circuit board according to the first circuit board image, according to the pin point coordinates and the pinhole point coordinates, the corresponding insertion point coordinates of the robot are determined, and according to the insertion point coordinates, the robot is controlled to insert the electronic component into the circuit board, through the embodiment of the invention, the insertion applicability of the electronic component is improved, complete automation can be realized, and the insertion accuracy of the electronic component can also be improved.

Description

Technical Field

[0001] The present invention relates to the field of automation technology, and in particular to an electronic component insertion method and device, product, equipment and medium. Background Art

[0002] With the continuous advancement of science and technology, the intelligence level and production efficiency of electronic equipment automated production lines are becoming increasingly higher.

[0003] The circuit boards of electronic devices need to integrate a variety of electronic components to achieve corresponding functions. Therefore, in the automated production process of electronic equipment, accurately inserting electronic components into the circuit boards by robot arms is an important part.

[0004] In the related art, the circuit board is positioned by identifying the Mark points (reference points, positioning points) of the circuit board, and then the robot arm is guided to insert the electronic components into the circuit board according to the positioning information. However, this solution is highly dependent on the recognition of the Mark points of the circuit board, and some circuit boards have Mark points printed on only one side, which makes this solution very limited and cannot be fully automated. In addition, the accuracy of inserting special-shaped electronic components is also relatively low. Summary of the invention

[0005] In view of the above problems, a method and device, product, equipment, and medium for inserting electronic components that overcome the above problems or at least partially solve the above problems are proposed, including:

[0006] A method for inserting an electronic component, the method comprising:

[0007] Acquire images of electronic components;

[0008] Determining the pin point coordinates of the electronic component according to the image of the electronic component;

[0009] According to the stitch point coordinates, determine the stitch spacing value, the number of stitches and the target angle value; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates;

[0010] Judging whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value;

[0011] If the electronic component is qualified, obtaining a first circuit board image of the circuit board, and determining the pinhole point coordinates of the circuit board according to the first circuit board image;

[0012] Determine the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates;

[0013] According to the insertion point coordinates, the robot is controlled to insert the electronic component into the circuit board.

[0014] Optionally, determining the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates includes:

[0015] Determine the first centroid point coordinates corresponding to the stitch point coordinates and the second centroid point coordinates corresponding to the pinhole point coordinates;

[0016] Performing a translation transformation on the stitch point coordinates according to the first centroid point coordinates to obtain a first target coordinate, and performing a translation transformation on the insertion point coordinates according to the second centroid point coordinates to obtain a second target coordinate;

[0017] Determining a covariance matrix according to the first target coordinates and the second target coordinates;

[0018] Determine the eigenvalues ​​and eigenvectors of the covariance matrix, and determine a rotation matrix based on the eigenvalues ​​and eigenvectors;

[0019] A translation vector is determined according to the rotation matrix, and the insertion point coordinates are determined according to the translation vector.

[0020] Optionally, determining the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates includes:

[0021] Obtaining a preset error equation; wherein the error equation represents the sum of squares of orthogonal distances from all pin points to the straight line model, and the error equation includes a slope parameter corresponding to the straight line model;

[0022] Determine a target slope value according to the error equation and the stitch point coordinates;

[0023] Determining the target angle value according to the target slope value;

[0024] The stitch spacing value and the stitch quantity are determined according to the stitch point coordinates.

[0025] Optionally, determining the stitch point coordinates of the electronic component according to the image of the electronic component includes:

[0026] Get the preset template image;

[0027] Matching the template image and the image of the electronic component to determine a first image region corresponding to the electronic component;

[0028] In the first image area, determining a second image area corresponding to the pins of the electronic component;

[0029] The stitch point coordinates are determined according to the second image area.

[0030] Optionally, before determining the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates, the method further includes:

[0031] Acquire a second circuit board image of the circuit board;

[0032] determining the coordinates of the reference point of the circuit board according to the second circuit board image;

[0033] Judging whether the circuit board is qualified according to the coordinates of the circuit board reference point and the coordinates of the pinhole point;

[0034] If the circuit board is qualified, the step of determining the coordinates of the corresponding insertion point of the robot according to the stitch point coordinates and the pinhole point coordinates is performed.

[0035] Optionally, the method further comprises:

[0036] In the process of controlling the robot to insert the electronic component into the circuit board, obtaining and saving the pressure data on the circuit board;

[0037] After controlling the robot to insert the electronic component into the circuit board, the method further includes:

[0038] It is determined whether the electronic component is successfully installed according to the pressure data.

[0039] Optionally, matching the template image with the image of the electronic component to determine a first image region corresponding to the electronic component includes:

[0040] The image of the electronic component is subjected to image preprocessing, and the template image and the image of the electronic component subjected to image preprocessing are matched to determine a first image region corresponding to the electronic component.

[0041] Optionally, after determining the target angle value according to the target slope value, the method further includes:

[0042] Determine the root mean square error and the coefficient of determination of the fitting straight line; wherein the root mean square error and the coefficient of determination are used to characterize the fitting quality index of the fitting straight line.

[0043] An electronic component insertion device, the device comprising:

[0044] An electronic component image acquisition module, used to acquire an image of the electronic component;

[0045] A stitch point coordinate determination module, used to determine the stitch point coordinates of the electronic component according to the image of the electronic component;

[0046] A target data determination module, used to determine the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates;

[0047] An electronic component qualification judgment module, used to judge whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value;

[0048] A pinhole point coordinate determination module, used for obtaining a first circuit board image of the circuit board if the electronic component is qualified, and determining the pinhole point coordinates of the circuit board according to the first circuit board image;

[0049] An insertion point coordinate determination module, used to determine the insertion point coordinates corresponding to the robot according to the pin point coordinates and the pinhole point coordinates;

[0050] The insertion control module is used to control the robot to insert the electronic component into the circuit board according to the insertion point coordinates.

[0051] A computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned electronic component insertion method.

[0052] An electronic device, characterized in that it includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the electronic component insertion method as described above is implemented.

[0053] A computer-readable storage medium is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the electronic component insertion method as described above is implemented.

[0054] The embodiments of the present invention have the following advantages: by acquiring an image of an electronic component, determining the coordinates of the pin point of the electronic component according to the image of the electronic component, determining the pin spacing value, the number of pins and the target angle value according to the pin point coordinates, wherein the target angle value is the angle value of the fitting straight line corresponding to the pin point coordinates, judging whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value, if the electronic component is qualified, acquiring a first circuit board image of the circuit board, determining the pinhole point coordinates of the circuit board according to the first circuit board image, determining the corresponding insertion point coordinates of the robot according to the pin point coordinates and the pinhole point coordinates, and controlling the robot to insert the electronic component into the circuit board according to the insertion point coordinates, thereby improving the applicability of electronic component insertion, realizing full automation, and improving the accuracy of electronic component insertion. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0056] Figure 1 It is a flowchart of the steps of an electronic component insertion method provided by one embodiment of the present invention;

[0057] Figure 2 A schematic diagram of pin status identification is provided according to an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of pin deflection angle identification is provided according to an embodiment of the present invention;

[0059] Figure 4 A schematic diagram of Mark point status recognition is provided in one embodiment of the present invention;

[0060] Figure 5 It is a structural schematic diagram of a special-shaped plug-in system provided by one embodiment of the present invention;

[0061] Figure 6 It is a schematic diagram of a special-shaped plug-in process provided by an embodiment of the present invention;

[0062] Figure 7 It is a schematic diagram of an insertion coordinate determination process provided by an embodiment of the present invention;

[0063] Figure 8 It is a structural block diagram of an electronic component insertion device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] With the support of hardware computing power, visual technology is widely used in automated production lines with robots for high-precision positioning and object feature detection. This technology reduces the number of personnel required for production lines and improves the degree of automation and production efficiency of production lines. Automated production lines have requirements such as high production yield, continuous production and high production efficiency. They need to be assisted by visual positioning or detection systems so that robots can accurately obtain targets for production operations such as insertion, clamping or spraying, thereby achieving fully automated functional requirements.

[0066] Special-shaped electronic components (or special-shaped plug-ins) refer to electronic components with irregular shapes that cannot be installed by standard automated equipment. These electronic components usually have complex shapes, multiple pins, special packaging or require manual operation.

[0067] In a related technology, the offset is calculated by obtaining the pixel coordinates of the special-shaped plug-in and the MARK point and comparing them with the template coordinates. After the insertion coordinates are output, the robot is guided to insert, so that the robot can grab the special-shaped plug-in and insert it continuously at a specific position on the PCB board. This solution can only realize the insertion on the PCB board with the MARK point, which has great limitations and cannot achieve full automation.

[0068] The embodiments of the present invention are described in detail below.

[0069] Reference Figure 1 , shows a flowchart of a method for inserting an electronic component provided by an embodiment of the present invention, which may specifically include the following steps:

[0070] Step 101, acquiring an image of an electronic component;

[0071] In step 101, the electronic components to be installed on the circuit board may also be called plug-ins, which may be passive components such as integrated circuits of resistors, capacitors, inductors, active components such as transistors, diodes, or special-shaped plug-ins such as connectors, sensors, and special packaging components.

[0072] In some examples, a high-precision camera may be provided on an automated production line to photograph electronic components to obtain images of the electronic components to be installed.

[0073] Step 102, determining the pin point coordinates of the electronic component according to the image of the electronic component;

[0074] In step 102, the stitch point is the image pixel point corresponding to each stitch of the electronic component, and the stitch point coordinate is the coordinate corresponding to the stitch point in the pixel coordinate system.

[0075] In some examples, the stitches can be identified by performing image recognition on images of electronic components, for example, using preset image algorithms, artificial intelligence models, etc., and extracting their feature points, which are used as stitch points to determine the stitch point coordinates of the electronic component.

[0076] In some embodiments of the present invention, step 102 includes:

[0077] Sub-step 1021, obtaining a preset template image;

[0078] Sub-step 1022, matching the template image with the image of the electronic component to determine a first image region corresponding to the electronic component;

[0079] Sub-step 1023, determining, in the first image area, a second image area corresponding to the pins of the electronic component;

[0080] Sub-step 1024, determining the stitch point coordinates according to the second image area.

[0081] In a specific implementation, the preset template image is a template image corresponding to the electronic component to be inserted, for example, an image of an electronic component of the same model.

[0082] The template image is matched with the image of the electronic component, and the actual area occupied by the electronic component is identified in the image of the electronic component, that is, the first image area is determined to narrow the range of pin identification. The first image area can be a geometric figure corresponding to the electronic component.

[0083] Then, from the first image area, the image area corresponding to the stitch is determined for recognition, further narrowing the scope of stitch recognition, that is, determining the second image area. The second image area can be a geometric figure corresponding to the stitch, or a frame covering the stitch image, etc.

[0084] After the second image area is determined, the stitch points corresponding to each stitch can be extracted in the second image area, and then the coordinates of the stitch points can be determined.

[0085] By continuously narrowing the range of stitches, the accuracy of stitch point recognition can be effectively improved.

[0086] In some examples, such as Figure 2 As shown, the electronic component is first matched with the template image to roughly locate the image area of ​​the electronic component, that is, to determine the first image area of ​​the electronic component, and then a ROI (Region of Interest) is selected in the first image area, and the ROI covers the image area corresponding to all pins of the electronic component (such as Figure 2A total of 3 pins are selected), which is the second image area.

[0087] Then, after performing affine transformation (geometric transformation of the image through affine matrix) and Blob analysis (blob analysis, blobs refer to connected areas in the image with similar features (such as color, grayscale, texture, etc.), and the goal of Blob analysis is to identify, segment and describe these areas for further analysis or processing) on ​​the ROI, the stitch points can be accurately positioned, and then the stitch point coordinates of each stitch can be determined to obtain a point set of stitch point coordinates.

[0088] In some embodiments of the present invention, sub-step 1022 further includes:

[0089] The image of the electronic component is subjected to image preprocessing, and the template image and the image of the electronic component subjected to image preprocessing are matched to determine a first image region corresponding to the electronic component.

[0090] In a specific implementation, the image of the electronic component may be subjected to graphic preprocessing such as binarization, filtering, and noise reduction to improve the recognition accuracy of the stitch point coordinates.

[0091] Step 103, determining the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates;

[0092] In step 103, the pin spacing value refers to the spacing value between every two pins, and the target angle value is the angle value of the fitting line corresponding to the coordinates of all pin points, which can reflect the overall angle deflection level of all pins of the electronic component, such as Figure 3 As shown. The pins of electronic components are fragile and easy to deform. If the pins are damaged or deformed, the number of pins, the pin spacing, and the pin angle deflection will not meet the factory standards, which will not only affect the success rate of insertion, but also affect the production quality of the product. By determining the pin spacing value, the number of pins, and the target angle value, the state of the pins can be detected to determine whether it is a qualified electronic component.

[0093] In some embodiments of the present invention, step 103 includes:

[0094] Sub-step 1031, obtaining a preset error equation; wherein the error equation represents the sum of squares of orthogonal distances from all stitch points to the straight line model, and the error equation includes a slope parameter corresponding to the straight line model;

[0095] Sub-step 1032, determining a target slope value according to the error equation and the stitch point coordinates;

[0096] Sub-step 1034, determining the target angle value according to the target slope value;

[0097] Sub-step 1035, determining the stitch spacing value and the stitch quantity according to the stitch point coordinates.

[0098] In some examples, the derivation process of the preset error equation can be as follows:

[0099] The linear model is constructed as y=mx+c, where m is the slope parameter and c is the intercept parameter;

[0100] Assume that the point set of the pin point coordinates is (xi, yi), where (i = 1, 2, ..., N), then the objective function d of the linear model is constructed i As shown below:

[0101]

[0102] This objective function represents the sum of squares from each pin point (xi, yi) to the fitting line;

[0103] Then the preset error equation E(m, c) can be expressed as follows:

[0104]

[0105] The error equation represents the sum of the squares of the orthogonal distances from all stitch points to the fitting line. Through the error equation, the fitting lines corresponding to the coordinates of all stitch points can be determined so that the sum of the squares of the orthogonal distances of the fitting lines of all stitch points is minimized. In this way, both the error in the Y direction and the error in the X direction are taken into account.

[0106] Then, the target slope value m is solved by using the point set of the above stitch point coordinates and combining the Levenberg-Marquardt algorithm to minimize the error equation. d and the target intercept value c d .

[0107] That is, the fitting straight line corresponding to the stitch point coordinates can be expressed as y=m d x+c d The target angle value θ directly corresponding to the fitting can be obtained by: arctan(m d ) is calculated, and the target angle value can reflect the overall deflection angle level of all pins.

[0108] In some embodiments of the present invention, after sub-step 1034, the method further includes:

[0109] Determine the root mean square error and the coefficient of determination of the fitting straight line; wherein the root mean square error and the coefficient of determination are used to characterize the fitting quality index of the fitting straight line.

[0110] In the specific implementation, the fitting straight line corresponding to the target angle value is assumed to be y=mx+c, and the root mean square error (RMSE) and the coefficient of determination (R-squared) of the fitting straight line model are calculated to obtain the fitting quality index of the fitting straight line model to reflect the quality of its fitting, so as to provide a reference for the subsequent iteration of the fitting straight line.

[0111] Step 104, judging whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value;

[0112] In some examples, the pin spacing value, the number of pins, and the target angle value can be compared with the preset threshold ranges corresponding to each data. If they all meet the preset threshold ranges, the electronic component can be considered qualified; if the electronic component is unqualified, the material is processed and the electronic component to be installed is re-grabbed.

[0113] Step 105, if the electronic component is qualified, obtaining a first circuit board image of the circuit board, and determining the pinhole point coordinates of the circuit board according to the first circuit board image;

[0114] In step 105, the pinhole point is the image pixel point corresponding to each pinhole of the circuit board in the first circuit board image, and the pinhole point coordinates are the coordinates of the pinhole point in the pixel coordinate system. The size of the pinhole of the circuit board (such as PCB board, printed circuit board) corresponds to the pin of the electronic component, and the insertion is successful by inserting all the pins of the electronic component into the pinholes corresponding to the circuit board.

[0115] In some examples, the first circuit board image may be a photographed image of the back side of the circuit board.

[0116] In some examples, the pinhole point coordinates are obtained by the following steps:

[0117] 1) Perform image preprocessing such as binarization and filtering on the image of the circuit board;

[0118] 2) using the template image corresponding to the circuit board to match the image of the circuit board to roughly locate the circuit board;

[0119] 3) Select the ROI area, perform blob analysis on the ROI area after affine transformation, and obtain the information of each pinhole point and the point set of the pinhole point coordinates.

[0120] Step 106, determining the corresponding insertion point coordinates of the robot according to the stitch point coordinates and the pinhole point coordinates;

[0121] In step 106, since the stitch point coordinates and the pinhole point coordinates are coordinates on the pixel coordinate system, it is also necessary to transfer the stitch point coordinates and the pinhole point coordinates from the pixel coordinates to the robot coordinate system through a certain conversion relationship, and then determine the insertion point coordinates for the robot to perform insertion identification.

[0122] In some embodiments of the present invention, step 106 includes:

[0123] Sub-step 1061, determining the first centroid coordinates corresponding to the stitch point coordinates and the second centroid coordinates corresponding to the pinhole point coordinates;

[0124] In a specific implementation, the point set of the pin point coordinates and the point set of the pinhole point coordinates can be cleaned first, for example, by using the ANSAC (random sampling consensus) algorithm to remove noise and abnormal points in the two point sets and replace them with standard points.

[0125] Then, the point set of the pin point coordinates and the pinhole point coordinates are converted from the pixel coordinate system to the robot coordinate system through the hand-eye calibration matrix. Hand-eye calibration refers to the process of determining the relative position and posture relationship between the robot's end effector (hand) and the camera (eye). The hand-eye calibration matrix is ​​used to describe this relative relationship.

[0126] Sub-step 1062, performing a translation transformation on the stitch point coordinates according to the first centroid point coordinates to obtain a first target coordinate, and performing a translation transformation on the insertion point coordinates according to the second centroid point coordinates to obtain a second target coordinate;

[0127] In a specific implementation, the centroid (geometric center) point coordinates of the point set of the stitch point coordinates, ie, the first centroid point coordinates, are determined, and the centroid point coordinates of the point set of the pinhole point coordinates, ie, the second centroid point coordinates, are determined.

[0128] Assume the coordinates of the pin point are pi, and the coordinates of the first centroid are pm. Then the coordinates of the pin point pi' after translation transformation are pi'=pi-pm, that is, the first target coordinates are obtained;

[0129] Assume that the coordinates of the pinhole point are qj and the coordinates of the first centroid point are qm. Then the coordinates of the pinpoint point qi' after translation transformation are qi'=qi-qm, that is, the second target coordinates are obtained.

[0130] In practical applications, a pinhole ROI area of ​​the insertion area can also be added to limit the range of physical movement to improve the alignment process.

[0131] Sub-step 1063, determining a covariance matrix according to the first target coordinates and the second target coordinates;

[0132] In the specific implementation, let pi' be the vector corresponding to the first target coordinates, qi' be the vector corresponding to the second target coordinates, then the covariance matrix C can be expressed as:

[0133]

[0134] Sub-step 1064, determining the eigenvalues ​​and eigenvectors of the covariance matrix, and determining a rotation matrix according to the eigenvalues ​​and eigenvectors;

[0135] In the specific implementation, the eigenvalue λ of the covariance matrix C is assumed to satisfy:

[0136] det(C-λI)=0

[0137] Where I is the identity matrix of the covariance matrix. Solving the above formula can obtain all eigenvalues: λ1,λ2,…,λ m ;

[0138] For each eigenvalue λ i , solving the following linear equations can get the corresponding eigenvector v i :

[0139] (C-λ i I)v i =0

[0140] Perform singular value decomposition (SVD) on the covariance matrix C:

[0141] C=USV T

[0142] Among them, U and V are orthogonal matrices, S is a diagonal matrix, and the diagonal elements of S are the singular values ​​of C;

[0143] Solve for singular values:

[0144] 1) Calculate C first T C and CC T :

[0145] C T C=VΛΛ T

[0146] CC T =UΣ 2 U T

[0147] 2) For C T C and CC T Solve for the eigenvector and eigenvalue to get C T The eigenvalue λ of C i and the eigenvector v i , the eigenvalue σi of CCT 2and the eigenvector u i , where σ i and λ i satisfy:

[0148]

[0149] 3) Construct the above matrices U, V, S: i Construct the matrix U as a column vector, and transform v i Construct the matrix V as a column vector and construct the diagonal matrix S, where the elements on the object line of S are σ i , which are the singular values ​​of the matrix C.

[0150] Through U and V, determine the rotation matrix R:

[0151] R=UV T

[0152] Sub-step 1065, determining a translation vector according to the rotation matrix, and determining the insertion point coordinates according to the translation vector.

[0153] In the specific implementation, let the rotation matrix be R, then the translation vector t is:

[0154]

[0155] in, Represents the coordinates of the target point in a certain coordinate system. Indicates the coordinates of the target point in another coordinate system.

[0156] Furthermore, based on the above translation vector, coordinate transformation (translation, rotation) is performed on the coordinates of the stitch point on the pixel coordinate system to obtain the coordinates of the insertion point.

[0157] In some embodiments of the present invention, before step 106, the method further includes:

[0158] Acquire a second circuit board image of the circuit board;

[0159] determining the coordinates of the reference point of the circuit board according to the second circuit board image;

[0160] Judging whether the circuit board is qualified according to the coordinates of the circuit board reference point and the coordinates of the pinhole point;

[0161] If the circuit board is qualified, the step of determining the coordinates of the corresponding insertion point of the robot according to the stitch point coordinates and the pinhole point coordinates is performed.

[0162] In the specific implementation, the coordinates of the reference point of the circuit board are the coordinates of the Mark point of the circuit board. By judging whether the coordinates of the Mark point deviate from the preset range, it can be confirmed whether the circuit board is qualified; if qualified, execute the next step; if unqualified, perform the material transfer operation and re-grab the circuit board to be inserted.

[0163] In some examples, such as Figure 4 As shown, the coordinates of the circuit board reference point can be determined through the following process:

[0164] 1) The second circuit board image is subjected to image preprocessing such as binarization and filtering

[0165] 2) Obtain a template image corresponding to the circuit board, such as an image of a circuit board with Mark points printed on it with the same signal, and use template image matching to perform rough positioning of the circuit board.

[0166] 3) Select the ROI area, perform affine transformation based on the template matching results, perform circle detection and precise positioning of the ROI area, and obtain the information of each mark point and the point set of the reference point coordinates of the circuit board.

[0167] In some examples, the second circuit board image may be an image of a circuit board with Mark points printed thereon, such as an image of a front side of the circuit board.

[0168] Step 107: Control the robot to insert the electronic component into the circuit board according to the insertion point coordinates.

[0169] In step 107, after the insertion point coordinates are obtained, the robot arm can be guided to insert the electronic components onto the circuit board through the insertion point coordinates.

[0170] In some embodiments of the present invention, the method further comprises:

[0171] In the process of controlling the robot to insert the electronic component into the circuit board, obtaining and saving the pressure data on the circuit board;

[0172] After step 107, the method further includes:

[0173] It is determined whether the electronic component is successfully installed according to the pressure data.

[0174] In some examples, a pressure sensor can be set at the gripper of the robot arm. When the electronic component is pushed or pressed into the target position, the pressure sensor detects the actual applied pressure value to obtain the pressure applied by the gripper to the circuit board through the inserted electronic component, and then determines the pressure received by the circuit board to save its pressure value.

[0175] Furthermore, the pressure value is compared with a preset threshold range:

[0176] If the actual pressure value falls within the preset threshold range, the electronic component is considered to have been successfully inserted.

[0177] If the actual pressure value is lower or higher than the preset threshold range, it may indicate that the electronic component is not inserted correctly or a problem occurs during the insertion process (for example, the electronic component is deformed or stuck, etc.).

[0178] In practical applications, the threshold range can be set according to the characteristics of the electronic components (such as hardness, weight, etc.) and the standard pressure range in the assembly process (for example, obtained through actual testing).

[0179] In some examples of the present invention, a special-shaped plug-in system is also provided, which is mainly composed of a visual industrial computer, a robot controller, a camera, and a SCARA robot. Figure 5 As shown, the specific advantages are as follows:

[0180] 1) Install the upper camera (mobile camera) and light source on the two-axis structure of the SCARA robot, increase the robot end grasping margin, and improve the adaptability of application scenarios;

[0181] 2) Install the lower camera (fixed camera) and light source next to the insertion point of the circuit board to effectively reduce the insertion path length.

[0182] 3) When grabbing multiple special-shaped plug-ins continuously, the lower camera can collect the pin information of each plug-in, make an advance judgment on the status of each special-shaped plug-in, and then realize the processing of throwing and re-grabbing the material.

[0183] In combination with the special-shaped plug-in system, some embodiments of the present invention also provide a special-shaped component insertion process, such as Figure 6 The specific implementation process is as follows:

[0184] 1) First, perform an overall self-check on the visual system to ensure that the communication, camera and other connections are normal;

[0185] 2) The PCB board inflow and loading mechanism delivers the material to a specific location;

[0186] 3) The SCARA robot continuously grabs the material and moves it to the photo taking point to obtain the image of the material pins;

[0187] 4) Preprocess the stitch image and detect the stitch status, mainly the number of stitches, stitch spacing, and stitch deflection angle. Finally, output the pixel coordinates of each stitch and the stitch detection result. If the stitch detection of the material fails, re-grab the material for inspection.

[0188] 5) The robot moves to the mark point or pinhole point shooting position to obtain image information. If it is the front side of the PCB board, the mark point image is preprocessed, the mark point status is detected and the mark point pixel coordinates are output. If it is the back side of the PCB board, the pinhole image is preprocessed, the status of each pinhole is detected and the pixel coordinates of each pinhole are output.

[0189] Among them, the pin status and pinhole status of the electronic components are detected as follows: Figure 7 As shown, after the coordinates of the pin point and the pinhole point are transformed (the offset is calculated) by the above-mentioned translation vector, the coordinates of the insertion point can be obtained.

[0190] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0191] Reference Figure 8 , shows a schematic diagram of the structure of an electronic component insertion device provided by an embodiment of the present invention, which may specifically include the following modules:

[0192] The electronic component image acquisition module 801 is used to acquire the image of the electronic component;

[0193] A stitch point coordinate determination module 802, configured to determine the stitch point coordinates of the electronic component according to the image of the electronic component;

[0194] The target data determination module 803 is used to determine the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates;

[0195] An electronic component qualification judgment module 804 is used to judge whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value;

[0196] A pinhole point coordinate determination module 805 is used to obtain a first circuit board image of the circuit board if the electronic component is qualified, and determine the pinhole point coordinates of the circuit board according to the first circuit board image;

[0197] The insertion point coordinate determination module 806 is used to determine the insertion point coordinates corresponding to the robot according to the pin point coordinates and the pinhole point coordinates;

[0198] The insertion control module 807 is used to control the robot to insert the electronic component into the circuit board according to the insertion point coordinates.

[0199] In some embodiments of the present invention, the device further comprises:

[0200] A centroid determination module, used to determine the first centroid point coordinates corresponding to the stitch point coordinates and the second centroid point coordinates corresponding to the pinhole point coordinates;

[0201] a target coordinate determination module, configured to perform a translation transformation on the stitch point coordinates according to the first mass center point coordinates to obtain a first target coordinate, and perform a translation transformation on the insertion point coordinates according to the second mass center point coordinates to obtain a second target coordinate;

[0202] A covariance matrix determination module, used to determine a covariance matrix according to the first target coordinates and the second target coordinates;

[0203] A rotation matrix determination module, used to determine the eigenvalues ​​and eigenvectors of the covariance matrix, and determine the rotation matrix according to the eigenvalues ​​and eigenvectors;

[0204] The insertion point coordinate determination module is used to determine the translation vector according to the rotation matrix, and determine the insertion point coordinates according to the translation vector.

[0205] In some embodiments of the present invention, the device comprises:

[0206] An error equation acquisition module, used to acquire a preset error equation; wherein the error equation represents the sum of squares of orthogonal distances from all pin points to the straight line model, and the error equation includes a slope parameter corresponding to the straight line model;

[0207] A target slope value determination module, used to determine a target slope value according to the error equation and the stitch point coordinates;

[0208] A target angle value determination module, used to determine the target angle value according to the target slope value;

[0209] The stitch spacing value and the stitch quantity determination module is used to determine the stitch spacing value and the stitch quantity according to the stitch point coordinates.

[0210] In some embodiments of the present invention, the device comprises:

[0211] A template image acquisition module, used to acquire a preset template image;

[0212] A first image region determination module, configured to match the template image with the image of the electronic component to determine a first image region corresponding to the electronic component;

[0213] A second image area determination module, used to determine, in the first image area, a second image area corresponding to the pins of the electronic component;

[0214] A stitch point coordinate determination module is used to determine the stitch point coordinates according to the second image area.

[0215] In some embodiments of the present invention, the device further comprises:

[0216] A second circuit board image acquisition module, used to acquire a second circuit board image of the circuit board;

[0217] A circuit board reference point coordinate determination module, used to determine the circuit board reference point coordinates according to the second circuit board image;

[0218] A circuit board qualification determination module, used to determine whether the circuit board is qualified according to the coordinates of the circuit board reference point and the coordinates of the pinhole point;

[0219] The release module is used to execute the step of determining the corresponding insertion point coordinates of the robot according to the stitch point coordinates and the pinhole point coordinates if the circuit board is qualified.

[0220] In some embodiments of the present invention, the device further comprises:

[0221] A pressure data storage module, used for acquiring and storing the pressure data on the circuit board during the process of controlling the robot to insert the electronic component into the circuit board;

[0222] The insertion success judgment module is used to judge whether the electronic component is inserted successfully according to the pressure data.

[0223] In some embodiments of the present invention, the device further comprises:

[0224] The preprocessing module is used to perform image preprocessing on the image of the electronic component, and to match the template image with the image of the electronic component after image preprocessing, so as to determine a first image region corresponding to the electronic component.

[0225] In some embodiments of the present invention, the device further comprises:

[0226] The fitting quality index determination module is used to determine the root mean square error and the determination coefficient of the fitting straight line; wherein the root mean square error and the determination coefficient are used to characterize the fitting quality index of the fitting straight line.

[0227] Some embodiments of the present invention further provide an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the computer program implements the insertion of the above electronic components when executed by the processor.

[0228] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned electronic component insertion method is implemented.

[0229] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above electronic component insertion method when executed by a processor.

[0230] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0232] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0233] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0234] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0235] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0236] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0237] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0238] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0239] The above is a detailed introduction to the provided electronic component insertion method and device, product, equipment, and medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for inserting an electronic component, characterized in that: The method comprises: Acquire images of electronic components; Determining the pin point coordinates of the electronic component according to the image of the electronic component; According to the stitch point coordinates, determine the stitch spacing value, the number of stitches and the target angle value; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates; Judging whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value; If the electronic component is qualified, obtaining a first circuit board image of the circuit board, and determining the pinhole point coordinates of the circuit board according to the first circuit board image; Determine the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates; According to the insertion point coordinates, the robot is controlled to insert the electronic component into the circuit board.

2. The method according to claim 1, characterized in that The step of determining the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates comprises: Determine the coordinates of a first centroid point corresponding to the stitch point coordinates and the coordinates of a second centroid point corresponding to the pinhole point coordinates; Performing a translation transformation on the stitch point coordinates according to the first centroid point coordinates to obtain a first target coordinate, and performing a translation transformation on the insertion point coordinates according to the second centroid point coordinates to obtain a second target coordinate; Determining a covariance matrix according to the first target coordinates and the second target coordinates; Determine the eigenvalues ​​and eigenvectors of the covariance matrix, and determine a rotation matrix based on the eigenvalues ​​and eigenvectors; A translation vector is determined according to the rotation matrix, and the insertion point coordinates are determined according to the translation vector.

3. The method according to claim 1, characterized in that Determining the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates includes: Obtaining a preset error equation; wherein the error equation represents the sum of squares of orthogonal distances from all pin points to the straight line model, and the error equation includes a slope parameter corresponding to the straight line model; Determine a target slope value according to the error equation and the stitch point coordinates; Determining the target angle value according to the target slope value; The stitch spacing value and the stitch quantity are determined according to the stitch point coordinates.

4. The method according to claim 1, characterized in that: Determining the pin point coordinates of the electronic component according to the image of the electronic component includes: Get the preset template image; Matching the template image and the image of the electronic component to determine a first image region corresponding to the electronic component; In the first image area, determining a second image area corresponding to the pins of the electronic component; The stitch point coordinates are determined according to the second image area.

5. The method according to claim 1, characterized in that: Before determining the coordinates of the insertion point corresponding to the robot according to the stitch point coordinates and the pinhole point coordinates, the method further includes: Acquire a second circuit board image of the circuit board; determining the coordinates of the reference point of the circuit board according to the second circuit board image; Judging whether the circuit board is qualified according to the coordinates of the circuit board reference point and the coordinates of the pinhole point; If the circuit board is qualified, the step of determining the coordinates of the corresponding insertion point of the robot according to the stitch point coordinates and the pinhole point coordinates is performed.

6. The method according to claim 1, characterized in that The method further comprises: In the process of controlling the robot to insert the electronic component into the circuit board, obtaining and saving the pressure data on the circuit board; After controlling the robot to insert the electronic component into the circuit board, the method further includes: It is determined whether the electronic component is successfully installed according to the pressure data.

7. The method according to claim 4, characterized in that The matching of the template image and the image of the electronic component to determine a first image region corresponding to the electronic component includes: The image of the electronic component is subjected to image preprocessing, and the template image and the image of the electronic component subjected to image preprocessing are matched to determine a first image region corresponding to the electronic component.

8. The method according to claim 3, characterized in that After determining the target angle value according to the target slope value, the method further includes: Determine the root mean square error and the coefficient of determination of the fitting straight line; wherein the root mean square error and the coefficient of determination are used to characterize the fitting quality index of the fitting straight line.

9. An electronic component insertion device, characterized in that: The device comprises: An electronic component image acquisition module, used to acquire an image of the electronic component; A stitch point coordinate determination module, used to determine the stitch point coordinates of the electronic component according to the image of the electronic component; A target data determination module, used to determine the stitch spacing value, the number of stitches and the target angle value according to the stitch point coordinates; wherein the target angle value is the angle value of the fitting straight line corresponding to the stitch point coordinates; An electronic component qualification judgment module, used to judge whether the electronic component is qualified according to the pin spacing value, the number of pins and the target angle value; A pinhole point coordinate determination module, used for obtaining a first circuit board image of the circuit board if the electronic component is qualified, and determining the pinhole point coordinates of the circuit board according to the first circuit board image; An insertion point coordinate determination module, used to determine the insertion point coordinates corresponding to the robot according to the pin point coordinates and the pinhole point coordinates; The insertion control module is used to control the robot to insert the electronic component into the circuit board according to the insertion point coordinates.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method for inserting an electronic component according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the electronic component insertion method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic component insertion method according to any one of claims 1 to 8 is implemented.

Images