Visual processing-based insertion methods, apparatus, computer devices, and storage media

Abstract

This invention provides a vision-based insertion method, apparatus, computer device, and storage medium. The method includes parsing an image of a first component to obtain first coordinates of each first element feature on the first component image; parsing an image of a second component to obtain second coordinates of each second element feature on the second component image; converting the first and second coordinates into third and fourth coordinates based on a robot coordinate system, respectively; performing linear fitting between the third and fourth coordinates on the robot coordinate system to calculate the deviation between the third and fourth coordinates; calculating the robot's insertion compensation amount based on the deviation; and inserting the first and second components based on the insertion compensation amount. By performing linear fitting between the coordinates of the first and second element features, the deviation between the two coordinates is accurately calculated, and the robot's insertion position is compensated, achieving precise insertion of irregularly shaped components onto the circuit board.

Description

Technical Field

[0001] This invention relates to the field of visual processing technology, and in particular to a visual processing-based insertion method, apparatus, computer device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology and the support of computing power in hardware devices, vision technology, in conjunction with robots for high-precision positioning and object feature detection, has been widely applied in automated production lines. This technology reduces the personnel required for production lines and improves automation and production efficiency. Because automated production lines require high yield rates, continuous production, and high production efficiency, they need to be supported by vision-based positioning or detection systems. This allows robots to accurately acquire targets and perform production operations such as insertion, gripping, or spraying, thereby achieving the aforementioned functional requirements.

[0003] Traditional high-precision irregular-shaped component insertion systems are technologies applied in PCB board production. They primarily use two cameras to acquire and process images of the irregular-shaped component pins and PCB board solder holes, ultimately sending the resulting positioning information to a robot. This allows the robot to accurately pick up and insert the irregular-shaped component into specific locations on the PCB board. While this method can achieve high-precision insertion of various irregular-shaped components into the PCB board, its insertion accuracy is relatively low. Summary of the Invention

[0004] Therefore, it is necessary to provide a visual processing-based insertion method, apparatus, computer device, and storage medium to address the aforementioned technical problems.

[0005] A vision-based instrumentation method includes:

[0006] Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0007] The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0008] The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively;

[0009] The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated.

[0010] The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0011] In one embodiment, the step of parsing the first component image to obtain the coordinates of the first element feature on the first component image as the first coordinate, and parsing the second component image to obtain the coordinates of the second element feature on the second component image as the second coordinate includes:

[0012] The image of the first component is analyzed to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features;

[0013] The image of the second component is analyzed to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element;

[0014] A first template region of interest is obtained, and a first region of interest of the first component image is obtained based on the reference point coordinates of the first component image. The first region of interest and the first template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each first element feature in the first template region of interest are finely positioned to obtain the first coordinates of each first element feature in the first region of interest.

[0015] A second template region of interest is obtained, and a second region of interest in the second component image is obtained based on the reference point coordinates of the second component image. The second region of interest and the second template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0016] In one embodiment, the step of parsing the first component image to obtain the reference point coordinates of the first component image and the coarse positioning coordinates of each of the first element features includes:

[0017] Obtain a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image.

[0018] The step of parsing the image of the second component to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element includes:

[0019] Obtain the second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

[0020] In one embodiment, the step of obtaining the first region of interest of the first component image based on the reference point coordinates of the first component image includes:

[0021] Align the reference point coordinates of the first component image with the reference point coordinates of the first template image, and obtain the first region of interest of the first component image based on the position of the region of interest of the first template on the first template image according to the affine transformation.

[0022] The step of obtaining the second region of interest of the second component image based on the reference point coordinates of the second component image includes:

[0023] Align the reference point coordinates of the second component image with the reference point coordinates of the second template image, and obtain the second region of interest of the second component image based on the position of the region of interest of the second template on the second template image according to the affine transformation.

[0024] In one embodiment, the feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine localization on the coarse localization coordinates of each first element feature within the first template region of interest to obtain the first coordinates of each first element feature within the first region of interest, including:

[0025] The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine positioning on the coarse positioning coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest.

[0026] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest, and performs fine localization on the coarse localization coordinates of each second element feature within the second template region of interest to obtain the second coordinates of each second element feature within the second region of interest. The steps include:

[0027] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest. Based on the position of the template element in the second template region of interest, the coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0028] In one embodiment, the step of fitting the third coordinate and the fourth coordinate to a straight line in the robot coordinate system includes:

[0029] The third coordinate and the fourth coordinate are linearly fitted in the robot coordinate system using an empirical harmonic method.

[0030] In one embodiment, the step of calculating the insertion compensation amount based on the deviation amount includes:

[0031] Detect whether the deviation is greater than a preset deviation.

[0032] When the deviation is greater than the preset deviation, the first component and / or the second component are determined to be waste materials.

[0033] When the deviation is less than or equal to the preset deviation, the insertion compensation amount of the robot is calculated based on the deviation, and the first component and the second component are inserted based on the insertion compensation amount.

[0034] A vision-based insertion device, comprising:

[0035] The image acquisition module is used to acquire images of the first component and obtain an image of the first component; and to acquire images of the second component and obtain an image of the second component.

[0036] The coordinate acquisition module is used to parse the first component image to obtain the coordinates of each first element feature on the first component image as the first coordinate, and to parse the second component image to obtain the coordinates of each second element feature on the second component image as the second coordinate;

[0037] A coordinate transformation module is used to convert the first coordinate and the second coordinate into a third coordinate and a fourth coordinate based on the robot coordinate system, respectively.

[0038] The deviation calculation module is used to perform a straight line fit between the third coordinate and the fourth coordinate on the robot coordinate system and calculate the deviation between the third coordinate and the fourth coordinate.

[0039] The compensation calculation module is used to calculate the insertion compensation amount of the robot based on the deviation amount, and to insert the first component and the second component based on the insertion compensation amount.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to perform the following steps:

[0041] Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0042] The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0043] The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively;

[0044] The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated.

[0045] The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0046] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0047] Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0048] The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0049] The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively;

[0050] The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated.

[0051] The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0052] The aforementioned vision-based insertion method, apparatus, computer equipment, and storage medium can accurately calculate the deviation between the coordinates of the first element feature on the first component and the coordinates of the second element feature on the second component by performing linear fitting, thereby compensating for the insertion position of the robot and realizing the precise insertion of irregularly shaped plugs into the circuit board. Attached Figure Description

[0053] Figure 1 This is a schematic diagram illustrating an application scenario of a vision-based insertion method in one embodiment;

[0054] Figure 2 This is a flowchart illustrating a vision-based insertion method in one embodiment;

[0055] Figure 3 This is a structural block diagram of a vision-based insertion device in one embodiment;

[0056] Figure 4 This is an internal structural diagram of a computer device in one embodiment;

[0057] Figure 5A This is a schematic diagram of the image data splitting and mating process of a vision-based insertion method in one embodiment;

[0058] Figure 5B This is a schematic diagram of the system algorithm flow for a vision-based instrumentation method in one embodiment;

[0059] Figure 5C This is a schematic diagram of the system processing logic of a vision-based instrumentation method in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] Example 1

[0062] The visual processing-based insertion method provided in this application can be applied to, for example... Figure 1In the application scenario shown, the visual robot system includes a vision controller 1, a pin detection camera 9, a solder hole detection camera 4, and a light source 7. The vision controller is responsible for the implementation of vision algorithms and strategies, camera control, and robot communication. The pin detection camera is used to acquire images of the plug-in components and is applied to the pin detection of different irregularly shaped plug-in components 8. The solder hole detection camera is used to acquire images of the PCB (Printed Circuit Board) board 3 and is applied to the detection of solder holes that need to be inserted into the PCB board 3. The light source 7 provides illumination and is used to separate the pins and solder holes from the background to obtain the feature information of the pins and solder holes.

[0063] The PCB board is conveyed on the production line 2 and passes through the solder hole inspection camera 4. The production line 2 consists of a central control 10 and a conveyor belt 11. The central control 10 is responsible for controlling the entire production line and moving different irregular plug-in to the robot's gripping position. The conveyor belt 11 is responsible for transporting the PCB board to the designated insertion position.

[0064] The robot system includes robot 5 and fixture 6. Robot 5 and fixture 6 are responsible for picking up and transporting different irregularly shaped plug-ins to the top of pin detection camera 9 for continuous photography. After obtaining positioning information, the irregularly shaped plug-ins are continuously inserted into the designated positions on the PCB board, thereby inserting the irregularly shaped plug-ins onto the PCB.

[0065] In this embodiment, the vision controller 1 can be implemented using the computer described in the following embodiment. The vision controller acquires an image of the first component through a pin detection camera, and acquires an image of the second component through a weld hole detection camera. The first component image is analyzed to obtain the coordinates of each first element feature on the first component image as the first coordinate. The second component image is analyzed to obtain the coordinates of each second element feature on the second component image as the second coordinate. The first coordinate and the second coordinate are converted into a third coordinate and a fourth coordinate based on the robot coordinate system, respectively. The third coordinate and the fourth coordinate are fitted with a straight line on the robot coordinate system to calculate the deviation between the third coordinate and the fourth coordinate. The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0066] It should be understood that the vision-based insertion method in this application can be applied to scenarios where robots insert two parts. For ease of explanation, the following embodiments use an irregularly shaped plug-in as an example and a PCB circuit board as an example for further explanation. However, it should be understood that this example is only used to explain this application and is not intended to limit this application. It is equally applicable to other parts that can be inserted, and will not be described in detail in this embodiment.

[0067] Example 2

[0068] In this embodiment, as Figure 2 As shown, a vision-based instrumentation method is provided, which includes:

[0069] Step 210: Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0070] In this embodiment, the first component and the second component are interlocking components. For example, the first component is a shaped plug-in and the second component is a PCB circuit board. The shaped plug-in is provided with pins and the circuit board is provided with solder holes. Each pin is inserted into a solder hole.

[0071] In this embodiment, images of the first component and the second component are acquired by a camera or other photosensitive element, and images of the first component and the second component are obtained respectively.

[0072] Step 220: Analyze the first component image to obtain the coordinates of each first element feature on the first component image as the first coordinates; Analyze the second component image to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0073] In this embodiment, the image is parsed to obtain the position of the element feature on the image, that is, the coordinate of the element feature on the image. In one embodiment, the first element feature is a pin on a non-standard component, and the second element feature is a solder hole on a circuit board. In this embodiment, the first component image and the second component image are parsed respectively to obtain the position of the pin on the first component image, that is, the first coordinate, and to obtain the position of the solder hole on the second component image, that is, the second coordinate.

[0074] In one embodiment, the first component image is binarized, converting it into a black and white image. The binarized first component image is then analyzed to obtain the coordinates of each first element feature on the first component image as first coordinates. Similarly, the second component image is binarized, converting it into a black and white image. The binarized second component image is then analyzed to obtain the coordinates of each second element feature on the second component image as second coordinates. In this embodiment, binarizing the component images clearly displays element features. For example, when a light source illuminates a solder hole, the light passes through the solder hole and is captured by a photosensitive element, resulting in an image containing the solder hole. After binarization, the circuit board appears black, while the solder hole appears white. This facilitates obtaining the coordinates of element features and improves the accuracy of coordinate acquisition.

[0075] Step 230: Convert the first coordinate and the second coordinate into the third coordinate and the fourth coordinate based on the robot coordinate system, respectively.

[0076] In this embodiment, the first coordinate is converted into a third coordinate based on the robot coordinate system; the second coordinate is converted into a fourth coordinate based on the robot coordinate system. It should be understood that the coordinates of the element features in the obtained image in the above steps are teaching coordinates, and the robot coordinate system is a coordinate system established based on the robot's position and path. The robot can move at the workstation where the first component is placed and at the workstation where the second component is placed.

[0077] Step 240: Perform a straight-line fit between the third coordinate and the fourth coordinate on the robot coordinate system, and calculate the deviation between the third coordinate and the fourth coordinate.

[0078] It should be understood that during insertion, the irregularly shaped component is aligned with the circuit board, so that each pin on the irregularly shaped component aligns with a solder hole on the circuit board. Therefore, in this embodiment, the third coordinate of the pin and the fourth coordinate of the solder hole, which are in the same coordinate system, are linearly fitted one by one. Using the first coordinate point as the coincidence reference, the coordinates of each pin and the coordinate position of the solder hole are calculated, thereby calculating the deviation between the pin and the solder hole.

[0079] Step 250: Calculate the insertion compensation amount of the robot based on the deviation amount, and insert the first component and the second component based on the insertion compensation amount.

[0080] In this embodiment, the insertion compensation amount is the distance by which the robot adjusts its coordinates, used to adjust the distance after the robot grasps the irregularly shaped plug-in. This insertion compensation amount compensates for the robot's coordinates. The insertion compensation amount of the robot in the robot coordinates is calculated based on the deviation amount. The first and second components are inserted based on the robot coordinates and the insertion compensation amount. In this embodiment, the insertion compensation amount of the robot is calculated based on the overall deviation between the pin and the solder hole, so that the pins on the irregularly shaped plug-in grasped by the robot can be accurately aligned with the solder holes on the circuit board.

[0081] In the above embodiments, by performing linear fitting on the coordinates of the first element feature on the first component and the coordinates of the second element feature on the second component, the deviation between the two coordinates can be accurately calculated, thereby compensating for the robot's insertion position and realizing the precise insertion of irregularly shaped plugs into the circuit board.

[0082] In one embodiment, the step of parsing the first component image to obtain the coordinates of the first element feature on the first component image as the first coordinate, and parsing the second component image to obtain the coordinates of the second element feature on the second component image as the second coordinate includes:

[0083] The image of the first component is parsed to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features; the image of the second component is parsed to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each of the second element features.

[0084] A first template region of interest is obtained, and a first region of interest of the first component image is obtained based on the reference point coordinates of the first component image. The first region of interest and the first template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each first element feature in the first template region of interest are finely positioned to obtain the first coordinates of each first element feature in the first region of interest.

[0085] A second template region of interest is obtained, and a second region of interest in the second component image is obtained based on the reference point coordinates of the second component image. The second region of interest and the second template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0086] In this embodiment, the component image is first coarsely located to obtain the reference point coordinates of the first component image and the second component image, as well as the coarse location coordinates of each element feature. The reference point is used as the coordinate of the reference or origin of the coordinate system in the component image. For example, the reference point coordinates are the center coordinates.

[0087] The first template region of interest and the second template region of interest are pre-set regions of interest used as references, and the template region of interest is a pre-selected region of interest in the template image.

[0088] After obtaining the coarse positioning coordinates, the first and second regions of interest (ROIs) of the template are acquired. Based on the positions of the element features within the ROIs, a feature matching method based on the pattern contour is used to correct the obtained coarse positioning coordinates. Specifically, this method matches the element features within the ROIs of the template with the element features of the corresponding ROIs, and then corrects the coordinates by matching the positions of the element features within the ROIs with the positions of the corresponding ROIs, thereby obtaining the first and second precise positioning coordinates. In this embodiment, the positions of the pins and weld holes are first coarsely positioned to obtain coarse positioning coordinates. Then, the precise positioning coordinates of the pins and weld holes are obtained using the precise positioning of the ROIs. This effectively improves the accuracy of the coordinates, resulting in accurate element feature coordinates.

[0089] To obtain the coarse positioning coordinates of the element features, in one embodiment, the step of parsing the first component image to obtain the reference point coordinates of the first component image and the coarse positioning coordinates of each of the first element features includes:

[0090] Obtain a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image.

[0091] The step of parsing the image of the second component to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element includes:

[0092] Obtain the second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

[0093] In this embodiment, the first template image and the second template image are pre-acquired reference images. The first template image is a pre-acquired and stored image of an irregularly shaped component, and the second template image is a pre-acquired and stored image of a circuit board. The first and second template images serve as reference benchmarks, and accurate coordinates are recorded on both images. By comparing the first component image with the first template image, the center point coordinates of the first component image can be determined; similarly, by comparing the second component image with the second template image, the center point coordinates of the second component image can be determined. Based on the center point coordinates, the relative positions of each pin or solder hole to the center point coordinates can be obtained. Combining this with the positions of the element features on the first and second template images, the coarse positioning coordinates of each first element feature on the first component image and the coarse positioning coordinates of each second element feature on the second component image can be calculated.

[0094] In order to obtain the region of interest and achieve precise coordinate positioning, in one embodiment, the step of obtaining the first region of interest of the first component image based on the reference point coordinates of the first component image includes: aligning the reference point coordinates of the first component image with the reference point coordinates of the first template image, and obtaining the first region of interest of the first component image based on the position of the first template region of interest on the first template image according to an affine transformation.

[0095] The step of obtaining the second region of interest of the second component image based on the reference point coordinates of the second component image includes: aligning the reference point coordinates of the second component image with the reference point coordinates of the second template image, and obtaining the second region of interest of the second component image based on the position of the second template region of interest on the second template image according to an affine transformation.

[0096] In this embodiment, the first and second template regions of interest are regions of interest pre-selected on the first and second template images. Each region of interest has at least one element feature, and the coordinate positions of these element features are pre-measured and used as a reference for fine positioning. In this embodiment, the reference point coordinates of the component image and the reference point coordinates of the template image are aligned. Based on the position of the template region of interest on the template image, this position is mapped to the component image, thereby obtaining the position of the region of interest on the component image, and thus obtaining the region of interest on the component image.

[0097] In one embodiment, the feature matching method based on pattern contours compares the first region of interest and the first template region of interest, performs fine localization on the coarse localization coordinates of each first element feature within the first template region of interest, and obtains the first coordinates of each first element feature within the first region of interest, including:

[0098] The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine positioning on the coarse positioning coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest.

[0099] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest, and performs fine localization on the coarse localization coordinates of each second element feature within the second template region of interest to obtain the second coordinates of each second element feature within the second region of interest. The steps include:

[0100] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest. Based on the position of the template element in the second template region of interest, the coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0101] In this embodiment, the feature matching method based on pattern contours makes the similarity between pins and weld holes more sharply distributed, thereby improving the positioning accuracy. Specifically, a single pin is matched with pins in the first region of interest within the first region of interest, and a single weld hole is matched with weld holes in the second region of interest within the second region of interest within the second region of interest. This allows for precise positioning correction of the position of the pin in the first region of interest and the position of the weld hole in the second region of interest, thereby obtaining accurate first and second coordinates.

[0102] In one embodiment, the step of fitting the third coordinate and the fourth coordinate to a straight line in the robot coordinate system includes: fitting the third coordinate and the fourth coordinate to a straight line in the robot coordinate system using an empirical harmonic method.

[0103] In this embodiment, the third coordinate is the coordinate of the pin, and the fourth coordinate is the coordinate of the weld hole. The empirical harmonic method is used to reconcile the pin point set (x... i y i ) and weld hole set (x i y i Perform linear fitting, as shown in equation (1), and fit y i Multiplied by weighting factor w i Where ε is a small quantity, first with w i -0.5 y i As a new dependent variable, w i -0.5 and w i -0.5 x i As independent variables, b0 and b1 are obtained using the least squares method with an intercept of zero. As shown in equation (2), k in the fitting result is then... i Values ​​greater than k′2(0.95, 0.95) are considered possible outliers and are given a reduced weight. A second round of weighted fitting is performed to obtain the straight lines for the pin point set and the weld hole point set.

[0104]

[0105]

[0106] In one embodiment, the step of calculating the insertion compensation amount based on the deviation amount includes: detecting whether the deviation amount is greater than a preset deviation amount; when the deviation amount is greater than the preset deviation amount, determining that the first component and / or the second component are waste materials; when the deviation amount is less than or equal to the preset deviation amount, calculating the insertion compensation amount of the robot based on the deviation amount, and inserting the first component and the second component based on the insertion compensation amount.

[0107] In this embodiment, when the deviation is greater than the preset deviation, it means that the positional deviation between the pins on the image of the irregularly shaped plug-in and the solder holes on the circuit board image is large, making successful insertion impossible. In this case, the irregularly shaped plug-in or circuit board is determined to be waste material and discarded; the robot will not attempt to insert it. When the deviation is less than or equal to the preset deviation, it means that the positional deviation between the pins on the image of the irregularly shaped plug-in and the solder holes on the circuit board image is small. Therefore, the robot's motion coordinates are compensated, enabling the pins of the irregularly shaped plug-in to accurately align with the solder holes on the circuit board.

[0108] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0109] Example 3

[0110] In this embodiment, as Figure 1 The diagram shows the hardware components of a vision-based insertion system, including a vision system, a production line, and a robot system.

[0111] (1) The vision system includes a vision controller 1, a pin detection camera 9, a solder hole detection camera 4 and a light source 7. The vision controller is responsible for the implementation of vision algorithms and strategies, camera control and robot communication. The pin detection camera is used for pin detection of different irregular plug-in 8. The solder hole detection camera is used for solder hole detection of PCB board 3 that needs to be inserted. The light source is used to separate the pins and solder holes from the background to obtain the feature information of the pins and solder holes.

[0112] (2) The production line includes a central control unit 10 and a conveyor belt 11. The central control unit is responsible for controlling the entire production line and moving different irregular plug-in to the robot's gripping position. The conveyor belt is responsible for transporting the PCB board to the designated insertion position.

[0113] (3) The robot system includes a robot 5 and a fixture 6. The robot and fixture are responsible for picking up and transporting different irregular plug-in to the pin detection camera for continuous photography. After obtaining the positioning information, the irregular plug-in is continuously inserted into the designated position on the PCB board.

[0114] 2. For example Figure 5AThe image data splitting and coordination process is shown below. The main steps are as follows:

[0115] (1) The robot picks up the irregular plug-in 1 to the photo teaching point, and obtains the black and white image information provided by the pin detection camera in real time through the vision controller. It creates image variable 1 to store the image of the irregular plug-in 1 and saves the image number.

[0116] (2) The robot picks up the irregular plug-in 2 to the photo teaching point, and obtains the black and white image information provided by the pin detection camera in real time through the vision controller. It creates image variable 2 to store the image of the irregular plug-in 2 and saves the image number.

[0117] (3) The solder hole detection camera acquires PCB images and performs image segmentation. The images of insertion point 1 and insertion point 2 are cached using image variables 3 and 4.

[0118] (4) The vision controller extracts the corresponding pin detection image and weld hole detection image according to the image number;

[0119] (5) After processing by the visual positioning algorithm, the positioning deviation data is obtained to realize the continuous insertion of irregular plug-in 1 and 2.

[0120] 3. For example Figure 5B The diagram shows the overall algorithm framework for a generalized high-precision irregular plug-in system. The main processing steps are as follows:

[0121] (1) Using the image of the irregular plug-in and the image of the PCB board as input, coarse positioning is performed by template matching to obtain the pixel coordinates of the center of the irregular plug-in and the pixel coordinates of the center of the PCB solder hole set.

[0122] (2) Obtain the ROI region containing the pin point set and the ROI region containing the solder hole point set through affine transformation. Perform pin search and solder hole search fine positioning within the region to obtain the pixel coordinates of each pin point and solder hole point.

[0123] (3) Use the empirical reconciliation method to reconcile the pin point set and solder hole point set (x) i y i Perform linear fitting, as shown in equation (1), to fit y i Multiplied by weighting factor w i Where ε is a small quantity, first with w i -0.5 y i As a new dependent variable, w i -0.5 and w i -0.5 x i As independent variables, b0 and b1 are obtained using the least squares method with an intercept of zero. As shown in equation (2), k in the fitting result is then... iValues ​​greater than k′2 (0.95, 0.95) are considered possible outliers and are given a reduced weight. A second round of weighted fitting is performed to obtain the straight lines for the pin point set and the weld hole point set.

[0124]

[0125]

[0126] (4) The straight line of pin point set and the straight line of weld hole point set take the first coordinate point as the coincidence reference, calculate the overall deviation, and determine whether the weld hole can cover all pin points. If it is feasible, output the robot insertion deviation and perform the insertion operation. If it is not feasible, discard the material.

[0127] 4. For example Figure 5C The diagram shown is a flowchart of the construction process for a millimeter-level vision positioning system. The main steps are as follows:

[0128] (1) First, the robot picks up the irregular plug-in and teaches it to the photo taking point and the insertion point, and obtains the corresponding template image of the irregular plug-in and the template image of the PCB board insertion position.

[0129] (2) Perform multi-template matching on the images acquired by the pin detection camera, select the center coordinates of the template with the highest matching score, and assign image numbers to the corresponding plug-ins according to the template;

[0130] (3) The solder hole detection camera acquires and segments the PCB board image, selects the corresponding template according to the image number, performs template matching on the segmented specific area image, and obtains the pixel center coordinates of the solder hole point set;

[0131] (4) Perform affine transformation based on the center coordinates of the original template and the center coordinates obtained after matching to obtain the ROI region of the pin points of the irregular plug-in and the ROI region of the solder hole point set;

[0132] (5) The above ROI area is precisely located. The feature matching method based on pattern contour is used to make the similarity of pins and weld holes show a sharper distribution, thereby improving the positioning accuracy. A single pin is used as a template to perform template matching in the ROI area, and a single weld hole is used as a template to perform template matching in the weld hole ROI area, so as to obtain the precise pixel coordinates of the center of each pin point and pin hole point.

[0133] (6) Convert pixel coordinates to robot coordinates based on the teaching coordinates, and obtain pin deviation, weld hole deviation and robot insertion compensation amount;

[0134] (7) Perform linear fitting on the pin point set and the weld hole point set, and take the first weld hole coordinate point as the coincidence reference. If the weld hole point set can cover all pins, then perform the insertion work; otherwise, discard the material.

[0135] This embodiment proposes a vision-based, generalized, high-precision irregular-shaped plug-in system. This method is applicable to different types of irregular-shaped plug-ins, and can achieve continuous insertion of multiple irregular-shaped plug-ins using only two cameras, thereby enhancing system production efficiency and reducing setup costs. The main process is as follows:

[0136] (1) Set up a pin inspection camera and a PCB board solder hole inspection camera, and teach the robot at the photo point and insertion point of each irregular plug-in;

[0137] (2) The robot picks up multiple different irregular plugs and takes continuous pictures at the pin detection camera. Then, the image information of each irregular plug is split into data, and the weld hole detection camera acquires the weld hole image information that needs to be inserted.

[0138] (3) At the same time, the image information and weld hole information of each type of irregular plug are matched. Through algorithms such as template matching coarse positioning, pin finding fine positioning, weld hole finding fine positioning and insertion detection judgment, the insertion positioning information deviation of each type of irregular plug is obtained and sent to the robot to realize the continuous insertion operation of different plugs.

[0139] Example 4

[0140] In this embodiment, as Figure 3 As shown, a vision-based insertion device is provided, comprising:

[0141] Image acquisition module 310 is used to acquire an image of the first component and obtain an image of the second component;

[0142] The coordinate acquisition module 320 is used to parse the first component image to obtain the coordinates of each first element feature on the first component image as the first coordinate, and to parse the second component image to obtain the coordinates of each second element feature on the second component image as the second coordinate;

[0143] The coordinate transformation module 330 is used to convert the first coordinate and the second coordinate into a third coordinate and a fourth coordinate based on the robot coordinate system, respectively.

[0144] The deviation calculation module 340 is used to perform a straight line fit between the third coordinate and the fourth coordinate on the robot coordinate system and calculate the deviation between the third coordinate and the fourth coordinate.

[0145] The compensation calculation module 350 is used to calculate the insertion compensation amount of the robot based on the deviation amount, and to insert the first component and the second component based on the insertion compensation amount.

[0146] In one embodiment, the coordinate acquisition module includes:

[0147] The first coarse positioning unit is used to parse the image of the first component to obtain the coordinates of the reference point of the image of the first component and the coarse positioning coordinates of each of the first element features.

[0148] The second coarse positioning unit is used to parse the image of the second component to obtain the coordinates of the reference point of the image of the second component and the coarse positioning coordinates of each feature of the second element.

[0149] The first fine localization unit is used to obtain the first template region of interest, and obtain the first region of interest of the first component image based on the reference point coordinates of the first component image. Based on the feature matching method of the pattern contour, the first region of interest and the first template region of interest are compared, and the coarse localization coordinates of each first element feature in the first template region of interest are finely localized to obtain the first coordinates of each first element feature in the first region of interest.

[0150] The second fine localization unit is used to obtain the region of interest of the second template and obtain the second region of interest of the second component image based on the reference point coordinates of the second component image. Based on the feature matching method of the pattern contour, the second region of interest and the second template region of interest are compared, and the coarse localization coordinates of each second element feature in the second template region of interest are finely localized to obtain the second coordinates of each second element feature in the second region of interest.

[0151] In one embodiment, the first coarse positioning unit is further configured to acquire a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image.

[0152] The first coarse positioning unit is further configured to acquire a second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

[0153] In one embodiment, the first precise positioning unit is further configured to align the reference point coordinates of the first component image and the reference point coordinates of the first template image, and obtain the first region of interest of the first component image based on the position of the first template region of interest on the first template image according to an affine transformation.

[0154] The second precision positioning unit is further configured to align the reference point coordinates of the second component image and the reference point coordinates of the second template image, and obtain the second region of interest of the second component image based on the position of the region of interest of the second template on the second template image according to the affine transformation.

[0155] In one embodiment, the first fine localization unit is further configured to compare the first region of interest and the first template region of interest using a feature matching method based on the pattern contour, and to perform fine localization on the coarse localization coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest.

[0156] The second fine localization unit is also used to compare the second region of interest and the second template region of interest based on the feature matching method of the pattern contour, and to perform fine localization on the coarse localization coordinates of each second element feature in the second template region of interest according to the position of the template element in the second template region of interest, so as to obtain the second coordinates of each second element feature in the second region of interest.

[0157] In one embodiment, the deviation calculation module is further configured to use an empirical harmonic method to perform a linear fit between the third coordinate and the fourth coordinate on the robot coordinate system.

[0158] In one embodiment, the compensation calculation module includes:

[0159] A deviation detection unit is used to detect whether the deviation is greater than a preset deviation.

[0160] The material throwing unit is used to determine that the first component and / or the second component are waste materials when the deviation is greater than the preset deviation.

[0161] An insertion unit is used to calculate the insertion compensation amount of the robot based on the deviation amount when the deviation amount is less than or equal to the preset deviation amount, and to insert the first component and the second component based on the insertion compensation amount.

[0162] Specific limitations regarding the vision-based instrumentation device can be found in the limitations of the vision-based instrumentation method described above, and will not be repeated here. Each unit in the aforementioned vision-based instrumentation device can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0163] Example 5

[0164] In this embodiment, a computer device is provided. Its internal structure diagram can be shown as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs, and also contains a database for storing template images and regions of interest. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with other computer devices that have deployed application software. When the computer program is executed by the processor, it implements a vision-based instrumentation method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0165] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0166] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0167] Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0168] The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0169] The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively;

[0170] The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated.

[0171] The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0172] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0173] The image of the first component is analyzed to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features;

[0174] The image of the second component is analyzed to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element;

[0175] A first template region of interest is obtained, and a first region of interest of the first component image is obtained based on the reference point coordinates of the first component image. The first region of interest and the first template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each first element feature in the first template region of interest are finely positioned to obtain the first coordinates of each first element feature in the first region of interest.

[0176] A second template region of interest is obtained, and a second region of interest in the second component image is obtained based on the reference point coordinates of the second component image. The second region of interest and the second template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0177] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0178] Obtain a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image.

[0179] Obtain the second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

[0180] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0181] Align the reference point coordinates of the first component image with the reference point coordinates of the first template image, and obtain the first region of interest of the first component image based on the position of the region of interest of the first template on the first template image according to the affine transformation.

[0182] Align the reference point coordinates of the second component image with the reference point coordinates of the second template image, and obtain the second region of interest of the second component image based on the position of the region of interest of the second template on the second template image according to the affine transformation.

[0183] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0184] The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine positioning on the coarse positioning coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest.

[0185] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest. Based on the position of the template element in the second template region of interest, the coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0186] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0187] The third coordinate and the fourth coordinate are linearly fitted in the robot coordinate system using an empirical harmonic method.

[0188] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0189] Detect whether the deviation is greater than a preset deviation.

[0190] When the deviation is greater than the preset deviation, the first component and / or the second component are determined to be waste materials.

[0191] When the deviation is less than or equal to the preset deviation, the insertion compensation amount of the robot is calculated based on the deviation, and the first component and the second component are inserted based on the insertion compensation amount.

[0192] Example 6

[0193] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps:

[0194] Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component.

[0195] The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates.

[0196] The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively;

[0197] The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated.

[0198] The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount.

[0199] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0200] The image of the first component is analyzed to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features;

[0201] The image of the second component is analyzed to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element;

[0202] A first template region of interest is obtained, and a first region of interest of the first component image is obtained based on the reference point coordinates of the first component image. The first region of interest and the first template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each first element feature in the first template region of interest are finely positioned to obtain the first coordinates of each first element feature in the first region of interest.

[0203] A second template region of interest is obtained, and a second region of interest in the second component image is obtained based on the reference point coordinates of the second component image. The second region of interest and the second template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0204] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0205] Obtain a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image.

[0206] Obtain the second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

[0207] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0208] Align the reference point coordinates of the first component image with the reference point coordinates of the first template image, and obtain the first region of interest of the first component image based on the position of the region of interest of the first template on the first template image according to the affine transformation.

[0209] Align the reference point coordinates of the second component image with the reference point coordinates of the second template image, and obtain the second region of interest of the second component image based on the position of the region of interest of the second template on the second template image according to the affine transformation.

[0210] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0211] The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine positioning on the coarse positioning coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest.

[0212] The feature matching method based on pattern contours compares the second region of interest and the second template region of interest. Based on the position of the template element in the second template region of interest, the coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

[0213] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0214] The third coordinate and the fourth coordinate are linearly fitted in the robot coordinate system using an empirical harmonic method.

[0215] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0216] Detect whether the deviation is greater than a preset deviation.

[0217] When the deviation is greater than the preset deviation, the first component and / or the second component are determined to be waste materials.

[0218] When the deviation is less than or equal to the preset deviation, the insertion compensation amount of the robot is calculated based on the deviation, and the first component and the second component are inserted based on the insertion compensation amount.

[0219] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0220] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0221] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A visual processing-based insertion method, characterized in that, include: Acquire an image of the first component to obtain an image of the first component; acquire an image of the second component to obtain an image of the second component. The first component image is parsed to obtain the coordinates of each first element feature on the first component image as the first coordinates; the second component image is parsed to obtain the coordinates of each second element feature on the second component image as the second coordinates. The first and second coordinates are converted into third and fourth coordinates based on the robot coordinate system, respectively; The third coordinate and the fourth coordinate are fitted to a straight line in the robot coordinate system, and the deviation between the third coordinate and the fourth coordinate is calculated. The insertion compensation amount of the robot is calculated based on the deviation amount, and the first component and the second component are inserted based on the insertion compensation amount; The steps of parsing the first component image to obtain the coordinates of the first element feature on the first component image as the first coordinate, and parsing the second component image to obtain the coordinates of the second element feature on the second component image as the second coordinate include: The image of the first component is analyzed to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features; The image of the second component is analyzed to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element; A first template region of interest is obtained, and a first region of interest of the first component image is obtained based on the reference point coordinates of the first component image. The first region of interest and the first template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each first element feature in the first template region of interest are finely positioned to obtain the first coordinates of each first element feature in the first region of interest. A second template region of interest is obtained, and a second region of interest in the second component image is obtained based on the reference point coordinates of the second component image. The second region of interest and the second template region of interest are compared based on the feature matching method of the pattern contour. The coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

2. The method according to claim 1, characterized in that, The step of parsing the image of the first component to obtain the coordinates of the reference point of the first component image and the coarse positioning coordinates of each of the first element features includes: Obtain a first template image, compare the first component image with the first template image to obtain the reference point coordinates of the first component image, and obtain the coarse positioning coordinates of each first element feature on the first component image based on the reference point coordinates of the first component image. The step of parsing the image of the second component to obtain the coordinates of the reference point of the second component image and the coarse positioning coordinates of each feature of the second element includes: Obtain the second template image, compare the second component image with the second template image to obtain the reference point coordinates of the second component image, and obtain the coarse positioning coordinates of each second element feature on the second component image based on the reference point coordinates of the second component image.

3. The method according to claim 2, characterized in that, The step of obtaining the first region of interest of the first component image based on the reference point coordinates of the first component image includes: Align the reference point coordinates of the first component image with the reference point coordinates of the first template image, and obtain the first region of interest of the first component image based on the position of the region of interest of the first template on the first template image according to the affine transformation. The step of obtaining the second region of interest of the second component image based on the reference point coordinates of the second component image includes: Align the reference point coordinates of the second component image with the reference point coordinates of the second template image, and obtain the second region of interest of the second component image based on the position of the region of interest of the second template on the second template image according to the affine transformation.

4. The method according to claim 1, characterized in that, The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine localization on the coarse localization coordinates of each first element feature within the first template region of interest to obtain the first coordinates of each first element feature within the first region of interest. The steps include: The feature matching method based on pattern contours compares the first region of interest and the first template region of interest, and performs fine positioning on the coarse positioning coordinates of each first element feature in the first template region of interest according to the position of the template element in the first template region of interest, so as to obtain the first coordinates of each first element feature in the first region of interest. The feature matching method based on pattern contours compares the second region of interest and the second template region of interest, and performs fine localization on the coarse localization coordinates of each second element feature within the second template region of interest to obtain the second coordinates of each second element feature within the second region of interest. The steps include: The feature matching method based on pattern contours compares the second region of interest and the second template region of interest. Based on the position of the template element in the second template region of interest, the coarse positioning coordinates of each second element feature in the second template region of interest are finely positioned to obtain the second coordinates of each second element feature in the second region of interest.

5. The method according to any one of claims 1-4, characterized in that, The step of fitting the third coordinate and the fourth coordinate to a straight line in the robot coordinate system includes: The third coordinate and the fourth coordinate are linearly fitted in the robot coordinate system using an empirical harmonic method.

6. The method according to any one of claims 1-4, characterized in that, The step of calculating the insertion compensation amount based on the deviation includes: Detect whether the deviation is greater than a preset deviation. When the deviation is greater than the preset deviation, the first component and / or the second component are determined to be waste materials. When the deviation is less than or equal to the preset deviation, the insertion compensation amount of the robot is calculated based on the deviation, and the first component and the second component are inserted based on the insertion compensation amount.

7. A visual processing-based insertion device, characterized in that, include: The image acquisition module is used to acquire images of the first component and obtain an image of the first component; and to acquire images of the second component and obtain an image of the second component. The coordinate acquisition module is used to parse the first component image to obtain the coordinates of each first element feature on the first component image as the first coordinate, and to parse the second component image to obtain the coordinates of each second element feature on the second component image as the second coordinate; A coordinate transformation module is used to convert the first coordinate and the second coordinate into a third coordinate and a fourth coordinate based on the robot coordinate system, respectively. The deviation calculation module is used to perform a straight line fit between the third coordinate and the fourth coordinate on the robot coordinate system and calculate the deviation between the third coordinate and the fourth coordinate. The compensation amount calculation module is used to calculate the insertion compensation amount of the robot based on the deviation amount, and to insert the first component and the second component based on the insertion compensation amount; The coordinate acquisition module includes: The first coarse positioning unit is used to parse the image of the first component to obtain the coordinates of the reference point of the image of the first component and the coarse positioning coordinates of each of the first element features. The second coarse positioning unit is used to parse the image of the second component to obtain the coordinates of the reference point of the image of the second component and the coarse positioning coordinates of each feature of the second element. The first fine localization unit is used to obtain the first template region of interest, and obtain the first region of interest of the first component image based on the reference point coordinates of the first component image. Based on the feature matching method of the pattern contour, the first region of interest and the first template region of interest are compared, and the coarse localization coordinates of each first element feature in the first template region of interest are finely localized to obtain the first coordinates of each first element feature in the first region of interest. The second fine localization unit is used to obtain the region of interest of the second template and obtain the second region of interest of the second component image based on the reference point coordinates of the second component image. Based on the feature matching method of the pattern contour, the second region of interest and the second template region of interest are compared, and the coarse localization coordinates of each second element feature in the second template region of interest are finely localized to obtain the second coordinates of each second element feature in the second region of interest.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images