A monocular vision-based pipelined workpiece distance measurement method and apparatus

Abstract

The application discloses a kind of pipeline workpiece distance measurement method, device and equipment based on monocular vision and computer storage medium, comprising: the image of workpiece to be measured photographed by camera is carried out distortion removal processing;Marker in the image after distortion removal is identified using deep learning network model;When identifying the marker in the processed image, the coordinates of the marker in the image are obtained, then the position coordinates are converted by image coordinates to control the camera to move to the position, and the marker image is photographed;Marker center is positioned using deep learning network to further judge whether the center of marker coincides with the center of the image;When the center of the marker in the image coincides with the center of the image, the position coordinates of the marker are obtained, and the pose and distance of the marker relative to the camera are calculated according to EPNP algorithm.The application identifies and locates the position of the marker by moving the camera to photograph the image, and then calculates the distance from the camera to the workpiece, so as to detect the workpiece and improve the efficiency of workpiece detection.

Description

Technical Field

[0001] This invention relates to the field of image positioning technology, and in particular to a method, apparatus, device, and computer-readable storage medium for measuring distances to workpieces on an assembly line based on monocular vision. Background Technology

[0002] The continuous development of computer technology and signal processing theory has given rise to many new research directions, one of which is machine vision. Machine vision is a technology that studies image signals and automatic control. Applying machine vision to the measurement of spatial geometric dimensions forms the concept of image measurement. Vision-based distance measurement methods and systems utilize image measurement to measure the distance between the target of interest and the camera, thereby providing a basis for system decision-making. Existing monocular vision-based distance measurement methods generally use a fixed-position camera and estimate the target depth using the pinhole imaging principle. If the detection of an object deviates, the camera position needs to be re-fixed, and it cannot simultaneously detect multiple devices.

[0003] Chinese invention patent application CN202011158472.1, entitled "A Method and System for Evaluating the Relative Accuracy Confidence of Monocular Vision Measurement at a Vehicle End," uses the correlation variance calculation of adjacent images to determine the three-dimensional position. Chinese invention patent application CN202010675208.9, entitled "A Method, Device, and Electronic Equipment for Monocular Vision Measurement of Robotic Arm Posture," uses a method of solving three-dimensional coordinates using fixed markers for pose measurement. In assembly line workpiece inspection, each workpiece needs to be inspected, and the distance between each workpiece and the inspection equipment needs to be measured so that the inspection equipment can move that distance to inspect the workpiece. The aforementioned monocular vision measurement methods all assume that the object being measured and the camera are fixed. However, in assembly line workpiece inspection, the measuring equipment needs to be moved for measurement. Therefore, the above patents are not applicable to assembly line workpiece inspection.

[0004] In summary, it can be seen that how to achieve moving distance measurement in workpiece inspection on an assembly line is a problem that needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, device, and computer-readable storage medium for measuring distances of workpieces on an assembly line based on monocular vision, which solves the drawback that existing distance measurement methods cannot be applied to assembly line inspection.

[0006] To address the aforementioned technical problems, this invention provides a method for measuring workpiece distance on an assembly line based on monocular vision, comprising:

[0007] S1: Perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image;

[0008] S2: Use a deep learning network model to identify landmarks in the distortion-free image;

[0009] S3: When a marker is identified in the image after distortion correction, the image coordinates of the marker in the image are obtained, the camera is controlled to move to the image coordinates of the marker in the image, and the image of the workpiece to be tested is re-captured and distortion correction is performed.

[0010] S4: Use the deep learning network model to determine whether the center of the marker in the image coincides with the center of the current distortion-free image;

[0011] S5: When the center of the marker in the image coincides with the center of the current distortion-free image, obtain the position coordinates of the marker on the workpiece to be tested, and calculate the pose information and distance information of the workpiece to be tested relative to the camera according to the EPNP algorithm.

[0012] Preferably, step S5 further includes:

[0013] Based on the pose information and the distance information, the detection device is controlled to move to the workpiece to be tested for detection.

[0014] Determine whether the device under test is powered on;

[0015] When the workpiece under test is powered on, the workpiece under test is considered qualified;

[0016] When the workpiece under test is not powered, the workpiece is considered unqualified and is marked.

[0017] Preferably, the step of performing distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain a distortion-corrected image includes:

[0018] The camera was calibrated using Zhang Zhengyou's calibration method to obtain the camera's intrinsic parameters and distortion parameters;

[0019] The camera is used to capture an image of the workpiece under test, thus obtaining an image of the workpiece under test;

[0020] The image of the workpiece under test is processed according to the distortion parameters to obtain the distortion-free image.

[0021] Preferably, the step of using a deep learning network model to identify markers in the distortion-free image further includes:

[0022] S21: When the marker is not present in the image after distortion correction, control the camera to move to find the marker and control the camera to re-photograph the workpiece to be tested;

[0023] S22: Re-capture the image of the workpiece under test with the camera, perform distortion correction processing to obtain a new distortion-corrected image, and return to step S2.

[0024] Preferably, the step of using the deep learning network model to locate whether the center of the marker in the image coincides with the center of the current distortion-corrected image further includes:

[0025] S41: When the center of the marker in the image does not coincide with the center of the current distorted image, obtain the new position coordinates of the marker in the image and control the camera to move to the new position coordinates;

[0026] S42: Control the camera to capture an image of the workpiece under test, perform distortion correction processing, and return to step S4.

[0027] Preferably, the step of calculating the pose and distance information of the workpiece relative to the camera according to the EPNP algorithm includes:

[0028] Based on the current position coordinates of the marker in the workpiece to be tested, construct the three-dimensional coordinates of the marker;

[0029] Construct the coordinates of virtual control points in the world coordinate system;

[0030] Calculate the coordinate relationship between the virtual control point coordinates and the three-dimensional coordinates of the marker in the workpiece to be measured;

[0031] Calculate the weighted sum of the coordinates of the markers and the coordinates of the virtual control points in the workpiece to be tested;

[0032] The relationship between each control point and the image point is calculated based on the weighted sum of the coordinates of the markers in the workpiece under test and the coordinates of the virtual control points, as well as the camera intrinsic parameters.

[0033] Calculate the distance between the four control points and the coordinates of the four control points in the camera coordinate system;

[0034] The pose and distance information of the marker in the workpiece to be measured to the camera are calculated based on the coordinates of the four control points in the camera coordinate system.

[0035] Preferably, both steps S2 and S4 use lightweight feature extraction networks and feature fusion graph operations to identify and locate landmarks in the image.

[0036] The present invention also provides a device for measuring the distance of workpieces on an assembly line based on monocular vision, comprising:

[0037] The image processing module is used to perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image;

[0038] The marker recognition module is used to identify markers in the distortion-free image using a deep learning network model;

[0039] The motion control module is used to obtain the image coordinates of the marker in the image when a marker is identified in the image after distortion correction, control the camera to move to the image coordinates of the marker in the image, re-capture the image of the workpiece under test and perform distortion correction processing;

[0040] The marker localization module is used to locate whether the center of a marker in the image coincides with the center of the current distortion-free image using the deep learning network model.

[0041] The distance calculation module is used to obtain the position coordinates of the marker on the workpiece under test when the center of the marker in the image coincides with the center of the current distortion-free image, and to calculate the pose information and distance information of the workpiece under test relative to the camera according to the EPNP algorithm.

[0042] The present invention also provides a device for measuring the distance between workpieces on an assembly line based on monocular vision, comprising:

[0043] A camera used to capture images according to control commands;

[0044] A mobile device that controls the camera to move according to control commands;

[0045] Memory, used to store computer programs;

[0046] The host computer is used to execute the computer program to implement the steps of the above-described method for measuring the distance of a workpiece on an assembly line based on monocular vision.

[0047] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for measuring distance of workpieces on a monocular vision-based assembly line.

[0048] This invention provides a method for measuring the distance of a workpiece on an assembly line based on monocular vision. It employs a camera to capture an image of the workpiece under test, and utilizes a deep learning network model to identify whether there are markers in the image, improving the accuracy of identification. When a marker is detected in the image, its image coordinates are obtained, and the camera is moved to the image coordinate position. The deep learning network model is used to locate the position of the marker in the workpiece under test, and then the EPNP algorithm is used to calculate the pose and distance information of the marker from the camera. The invention involves capturing an image of the workpiece under test, moving the entire camera and detection device, identifying the position information of the marker in the image, moving the entire camera and detection device to align the center point of the image with the center point of the marker in the image, and then using the EPNP algorithm to calculate the distance and pose information of the marker from the camera. Based on the distance and pose information, the movement distance of the detection device is obtained, and the device under test is then detected. This invention identifies and locates the position of a marker by moving a camera, and then measures the distance. This solves the drawback of the prior art, which requires both the camera and the object to be measured to be fixed in order to measure the distance. It realizes the automation of workpiece inspection on the production line. The identification, positioning and inspection of the workpiece are all integrated, which improves the efficiency of workpiece inspection on the production line. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 A flowchart illustrating a first specific embodiment of the monocular vision-based assembly line workpiece distance measurement method provided by the present invention;

[0051] Figure 2 A flowchart illustrating a second specific embodiment of the monocular vision-based assembly line workpiece distance measurement method provided by the present invention;

[0052] Figure 3 A flowchart illustrating the steps of the monocular vision-based assembly line workpiece distance measurement method provided in this invention;

[0053] Figure 4 This is a structural diagram of the CSPDarknet53-Tiny model of the present invention;

[0054] Figure 5 The training graph for the detection model;

[0055] Figure 6This is a structural block diagram of a device for measuring the distance between workpieces on an assembly line based on monocular vision, provided in an embodiment of the present invention. Detailed Implementation

[0056] The core of this invention is to provide a method for measuring the distance of workpieces on an assembly line based on monocular vision. It uses a deep learning network model for identification and localization, and then calculates the distance information according to the EPNP algorithm, thereby improving the efficiency of workpiece detection on the assembly line.

[0057] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Please refer to Figure 1 , Figure 1 This is a flowchart of a first specific embodiment of the monocular vision-based workpiece distance measurement method for production lines provided by the present invention; the specific operation steps are as follows:

[0059] Step S101: Perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image;

[0060] Step S102: Use a deep learning network model to identify markers in the distortion-free image;

[0061] S21: When the marker is not present in the image after distortion correction, control the camera to move to find the marker and control the camera to re-photograph the workpiece to be tested;

[0062] S22: Re-capture the image of the workpiece under test with the camera and perform distortion correction processing to obtain a new distorted image, then return to step S102.

[0063] Step S103: When a marker is identified in the image after distortion correction, the image coordinates of the marker in the image are obtained, the camera is controlled to move to the image coordinates of the marker in the image, and the image of the workpiece to be tested is re-captured and distortion correction is performed.

[0064] Step S104: Use the deep learning network model to determine whether the center of the marker in the image coincides with the center of the current distortion-free image;

[0065] S41: When the center of the marker in the image does not coincide with the center of the current distorted image, obtain the new position coordinates of the marker in the image and control the camera to move to the new position coordinates;

[0066] S42: Control the camera to capture an image of the workpiece under test, perform distortion correction processing, and return to step S104.

[0067] Step S105: When the center of the marker in the image coincides with the center of the current distorted image, obtain the position coordinates of the marker on the workpiece to be tested, and calculate the pose information and distance information of the workpiece to be tested relative to the camera according to the EPNP algorithm.

[0068] In this embodiment, the present invention uses a camera to capture images to identify and locate the position of a marker, obtaining image information from the marker. Then, it calculates the distance information from the marker to the camera using the EPNP algorithm, improving the accuracy of identification and positioning. The present invention identifies and locates the position of the marker by moving the camera and then performs distance measurement, overcoming the drawback of existing technologies where both the camera and the marker need to be fixed for distance measurement. This achieves automation of workpiece inspection on the production line, integrating workpiece identification, positioning, and inspection, thus improving the efficiency of workpiece inspection on the production line.

[0069] Please refer to Figure 2 and Figure 3 , Figure 2 A flowchart illustrating a second specific embodiment of the monocular vision-based assembly line workpiece distance measurement method provided by the present invention; Figure 3 The flowchart illustrates the steps of the monocular vision-based workpiece distance measurement method for production lines provided in this invention; the specific operation steps are as follows:

[0070] Step S201: Calibrate the camera according to Zhang Zhengyou's calibration method to obtain the camera's intrinsic parameters and distortion parameters;

[0071] Step S202: Use a camera to capture an image of the workpiece, and then use distortion parameters to perform distortion correction.

[0072] The camera needs a certain field of view to capture the markings of the workpiece of interest, and the camera should be parallel to the power plug-in device. Based on the actual situation, a certain model of camera is selected and installed in a pre-set fixed position. The horizontal and vertical distances between the center point of the camera and the center point of the power plug-in device at this position are 50mm and 11mm, respectively.

[0073] Image distortion can lead to deviations in image localization results. To minimize the impact of image distortion on localization results, Zhang Zhengyou's calibration method is used to calibrate the camera, thereby obtaining the image's intrinsic parameters and distortion parameters for distortion correction. Furthermore, the camera's intrinsic parameters are also crucial for visual measurements, directly used in distance measurement calculations.

[0074] Step S203: Identify whether there is a power port in the workpiece image;

[0075] Step S204: Obtain the image coordinates of the power port in the image, and control the camera to move to the position of the power port;

[0076] Step S205: Determine whether the center position of the power port in the positioning image coincides with the center position of the image;

[0077] The positions of workpieces on an assembly line are not fixed each time they stop, and the positions of the workpiece markers are also not fixed, requiring a certain strategy to find the markers. This patent uses scanning and deep learning-based methods to find and locate the workpiece markers, specifically the power port positions. The scanning method refers to scanning the area where the workpiece is located in all directions (up, down, left, and right) to find the marker. Once the marker is found, it is locked, and the distance from the center point of the marker to the camera is measured. Since the marker and the workpiece are on the same plane, meaning the distance from the workpiece to the camera is equal to the distance from the marker to the camera, this distance can be used as the distance from the camera to the workpiece.

[0078] When the camera reaches the far left of the area and moves to the far right at a speed of 50 mm per movement, the workpiece marker is identified and located after each movement. The identification and location methods utilize deep learning. Considering the limited computing resources of edge computing devices, a small network model is used for workpiece marker identification and location. Specifically, the CSPDarknet53-Tiny model is used for power port identification and location. The specific structure is as follows: Figure 4 As shown.

[0079] In training the detection model, this invention employs data augmentation techniques, including image saturation, exposure, and chroma adjustments. The input image size is set to 416*416. Model parameter optimization utilizes dynamic SGD with warm-up operations and pre-training to accelerate training. The model's IoU loss uses CIOU loss, and the detection bounding box NMS uses simple NMS. The changes in loss during model training and the map value on the test set are as follows: Figure 5 As shown in the figure, the loss gradually decreases with the increase of the number of training rounds, while the map value on the test set gradually increases with the increase of the number of training rounds, eventually stabilizing.

[0080] Once the marker is identified and located, the marker is locked and the center point of the image is aligned with the center point of the marker. At this point, the camera is considered to have reached the designated position and the position coordinates of the marker are obtained.

[0081] Step S206: Calculate the distance information from the power port to the camera according to the EPNP algorithm;

[0082] Step S207: Control the power plug-in device to move according to the distance information to complete the detection.

[0083] Based on the current position coordinates of the marker in the workpiece to be tested, construct the three-dimensional coordinates of the marker;

[0084] Construct the coordinates of virtual control points in the world coordinate system;

[0085] Calculate the coordinate relationship between the virtual control point coordinates and the three-dimensional coordinates of the marker in the workpiece to be measured;

[0086] Calculate the weighted sum of the coordinates of the markers and the coordinates of the virtual control points in the workpiece to be tested;

[0087] The relationship between each control point and the image point is calculated based on the weighted sum of the coordinates of the markers in the workpiece under test and the coordinates of the virtual control points, as well as the camera intrinsic parameters.

[0088] Calculate the distance between the four control points and the coordinates of the four control points in the camera coordinate system;

[0089] The pose and distance information of the marker in the workpiece to be measured to the camera are calculated based on the coordinates of the four control points in the camera coordinate system.

[0090] Based on the pose information and the distance information, the detection device is controlled to move to the workpiece to be tested for detection.

[0091] Determine whether the device under test is powered on;

[0092] When the workpiece under test is powered on, the workpiece under test is considered qualified;

[0093] When the workpiece under test is not powered, the workpiece is considered unqualified and is marked.

[0094] After obtaining the position coordinates of the marker, these coordinates are used as the coordinates of the image plane. For the marker coordinates, it is assumed that the origin of the world coordinate system is at the center point of the marker plane. Since the specifications of the marker are known—it is rectangular with a length of 30mm and a width of 23mm—the three-dimensional and two-dimensional coordinates of the four vertices of the marker are known. The three-dimensional coordinates are [-15.0, 11.5, 0], [15.0, 11.5, 0], [-15.0, -11.5, 0], [15.0, -11.5, 0]. The two-dimensional coordinates are the image plane coordinates of the four detected vertices of the marker.

[0095] The EPNP algorithm generally requires knowledge of three pairs of coplanar or four pairs of non-coplanar points corresponding to the world coordinate system and the image coordinate system, as well as the camera's intrinsic parameters and distortion parameters. Based on this algorithm, the pose and distance of the target point of interest with respect to the camera can be calculated. Assume the coordinates of the four vertices of the target workpiece marker in the world coordinate system are... The non-homogeneous coordinates of the four virtual control points in the world coordinate system are: The relationship between the coordinates of the four vertices of the marker and the coordinates of the virtual control point is as follows:

[0096]

[0097] From the invariance of linear relations under Euclidean transformation, we know that:

[0098]

[0099] In the formula: c is used to label the coordinates in the camera coordinate system; The last element of the homogeneous coordinate is 1, and the transformation coefficient satisfies... [a i1 ,a i2 ,a i3 ,a i4 ] T The weights are the coordinates of the marker vertices in Euclidean space based on the virtual control points, and each feature point can be represented as a weighted sum of four virtual control points, as shown in formula (3):

[0100]

[0101] Assume the image coordinates corresponding to the feature points are (u i ,v i If the camera intrinsic parameter matrix obtained by calibrating the camera using Zhang Zhengyou's calibration method is A, then:

[0102]

[0103] In the formula: s i f is the projected depth of the feature points; A is the intrinsic parameter matrix of the camera, f x f y u x v y These are the camera's internal parameters. From formula (4), we can see that... The correspondence between each feature point and image point is as follows:

[0104]

[0105]

[0106] When there are four spatial points corresponding to image points, a system of eight linear equations can be obtained, denoted as a matrix Mx = 0, where M is an 8×12 matrix and the vector x is a 12×1 vector containing the non-homogeneous coordinates of the four virtual control points in the camera coordinate system. Based on the distance-preserving property of Euclidean transformation, the distance between the four feature points is known, allowing the calculation of their coordinates in the camera coordinate system. This transforms the 3D-to-2D PNP problem into solving the classic 3D-to-3D rigid body motion problem, thereby obtaining the attitude and distance information of the target workpiece.

[0107] When the camera center is aligned with the center point of the marker, the coordinates of the marker's center point and its four vertices are obtained using a deep learning-based marker recognition and localization algorithm. Then, the rotation and translation matrices relative to the camera can be calculated using the EPNP algorithm. Since the camera and marker are parallel, the distance between the camera and marker is essentially the longitudinal distance in the relative pose, which can be obtained using the calculated rotation and translation matrices. The measurement results and actual values ​​are shown in Table 1. The table shows that the measured values ​​are close to the actual values, with the average value being very close with the actual values. The largest difference is 2.5 mm, and the smallest is 0.21 mm. The relative error is a minimum of 0.18% and a maximum of 1.68%. Furthermore, the data shows that the greater the distance between the workpiece and the camera, the greater the measurement error. This means that within the range of 12 cm to 14.87 cm between the workpiece and the camera, the maximum measurement error is only 2.5 mm, which meets production requirements.

[0108] Table 1 is a comparison table of the actual distance and the measured distance of the present invention.

[0109]

[0110] This invention facilitates communication between the algorithm and software sides via MQTT messages in JSON format. The software side drives the camera to move cyclically within the area where the workpiece is located on the production line. Simultaneously, a deep learning algorithm identifies markers and obtains their center coordinates. After conversion, these coordinates are sent to the software side via MQTT, and the software side reacts accordingly. When the moving axis reaches a designated position, i.e., when the camera center coincides with the marker center, the algorithm sends a command to the software side via MQTT to stop the moving axis.

[0111] Please refer to Figure 6 , Figure 6 This invention provides a structural block diagram of a device for measuring workpiece distance on an assembly line based on monocular vision; the specific device may include:

[0112] The image processing module 100 is used to perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image;

[0113] The marker recognition module 200 is used to identify markers in the distortion-free image using a deep learning network model.

[0114] The motion control module 300 is used to obtain the image coordinates of the marker in the image when a marker is identified in the image after distortion correction, control the camera to move to the image coordinates of the marker in the image, re-capture the image of the workpiece to be tested, and perform distortion correction processing.

[0115] The marker localization module 400 is used to locate whether the center of a marker in the image coincides with the center of the current distortion-free image using the deep learning network model.

[0116] The distance calculation module 500 is used to obtain the position coordinates of the marker on the workpiece under test when the center of the marker in the image coincides with the center of the current distortion-free image, and to calculate the pose information and distance information of the workpiece under test relative to the camera according to the EPNP algorithm.

[0117] This embodiment provides a device for measuring the distance of workpieces on a production line based on monocular vision. This device is used to implement the aforementioned method for measuring the distance of workpieces on a production line based on monocular vision. Therefore, the specific implementation of the device for measuring the distance of workpieces on a production line based on monocular vision can be found in the embodiment section of the method for measuring the distance of workpieces on a production line based on monocular vision described above. For example, the image processing module 100, the marker recognition module 200, the motion control module 300, the marker positioning module 400, and the distance calculation module 500 are used to implement steps S101, S102, S103, S104, and S105 in the aforementioned method for measuring the distance of workpieces on a production line based on monocular vision, respectively. Therefore, the specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0118] A specific embodiment of the present invention also provides a device for measuring the distance of workpieces on an assembly line based on monocular vision, comprising: a camera for capturing images according to control commands;

[0119] A mobile device that controls the camera to move according to control commands;

[0120] Memory, used to store computer programs;

[0121] The host computer is used to execute the computer program to implement the steps of the above-described method for measuring the distance of a workpiece on an assembly line based on monocular vision.

[0122] A specific embodiment of the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for measuring the distance of a workpiece on an assembly line based on monocular vision.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0124] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0125] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0126] The foregoing has provided a detailed description of a method, apparatus, device, and computer-readable storage medium for measuring workpiece distance on an assembly line based on monocular vision, as provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A method for measuring workpiece distance on an assembly line based on monocular vision, characterized in that, include: S1: Perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image; S2: Use a deep learning network model to identify landmarks in the distortion-free image; S3: When a marker is identified in the image after distortion correction, the image coordinates of the marker in the image are obtained, the camera is controlled to move to the image coordinates of the marker in the image, and the image of the workpiece to be tested is re-captured and distortion correction is performed. S4: Use the deep learning network model to determine whether the center of the marker in the image coincides with the center of the current distortion-free image; S5: When the center of the marker in the image coincides with the center of the current distorted image, obtain the position coordinates of the marker on the workpiece under test, and calculate the pose and distance information of the workpiece under test relative to the camera according to the EPNP algorithm, including: constructing the three-dimensional coordinates of the four vertices of the marker in the world coordinate system based on the current position coordinates of the marker in the workpiece under test; constructing the non-homogeneous coordinates of the four virtual control points in the world coordinate system; representing the coordinates of each vertex of the marker in the workpiece under test as a weighted sum of the coordinates of the four virtual control points; obtaining the relationship between each vertex of the marker and the four virtual control points in the camera coordinate system based on the invariance of linear relationships under Euclidean transformation; calculating the relationship between each virtual control point and the image point based on the image coordinates corresponding to the vertex coordinates of the marker in the workpiece under test, the weighted sum of the coordinates of the four virtual control points in the camera coordinate system, and the camera intrinsic parameters; calculating the distance between the four virtual control points and the coordinates of the four virtual control points in the camera coordinate system; and calculating the pose and distance information of the marker in the workpiece under test to the camera based on the coordinates of the four virtual control points in the camera coordinate system.

2. The distance measurement method as described in claim 1, characterized in that, Step S5 is followed by: Based on the pose information and the distance information, the detection device is controlled to move to the workpiece to be tested for detection. Determine whether the workpiece under test is powered on; When the workpiece under test is powered on, the workpiece under test is considered qualified; When the workpiece under test is not powered, the workpiece is considered unqualified and is marked.

3. The distance measurement method as described in claim 1, characterized in that, The process of performing distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image includes: The camera was calibrated using Zhang Zhengyou's calibration method to obtain the camera's intrinsic parameters and distortion parameters; The camera is used to capture an image of the workpiece under test, thus obtaining an image of the workpiece under test; The image of the workpiece under test is processed according to the distortion parameters to obtain the distortion-free image.

4. The distance measurement method as described in claim 1, characterized in that, The method of using a deep learning network model to identify markers in the distortion-removed image also includes: S21: When the marker is not present in the image after distortion correction, control the camera to move to find the marker and control the camera to re-photograph the workpiece to be tested; S22: Re-capture the image of the workpiece under test with the camera, perform distortion correction processing to obtain a new distortion-corrected image, and return to step S2.

5. The distance measurement method as described in claim 1, characterized in that, The step of using the deep learning network model to locate whether the center of the marker in the image coincides with the center of the current distortion-corrected image also includes: S41: When the center of the marker in the image does not coincide with the center of the current distorted image, obtain the new position coordinates of the marker in the image and control the camera to move to the new position coordinates; S42: Control the camera to capture an image of the workpiece under test, perform distortion correction processing, and return to step S4.

6. The distance measurement method as described in claim 1, characterized in that, Both steps S2 and S4 use lightweight feature extraction networks and feature fusion graph operations to identify and locate landmarks in the image.

7. A device for measuring workpiece distance on an assembly line based on monocular vision, characterized in that, include: The image processing module is used to perform distortion correction processing on the image of the workpiece to be tested captured by the camera to obtain the distortion-corrected image; The marker recognition module is used to identify markers in the distortion-free image using a deep learning network model; The motion control module is used to obtain the image coordinates of the marker in the image when a marker is identified in the image after distortion correction, control the camera to move to the image coordinates of the marker in the image, re-capture the image of the workpiece under test and perform distortion correction processing; The marker localization module is used to locate whether the center of a marker in the image coincides with the center of the current distortion-free image using the deep learning network model. The distance calculation module is used to obtain the position coordinates of the marker on the workpiece under test when the center of the marker in the image coincides with the center of the current distorted image, and to calculate the pose and distance information of the workpiece under test relative to the camera according to the EPNP algorithm. This includes: constructing the three-dimensional coordinates of the four vertices of the marker in the world coordinate system based on the current position coordinates of the marker in the workpiece under test; constructing the non-homogeneous coordinates of four virtual control points in the world coordinate system; representing the coordinates of each vertex of the marker in the workpiece under test as a weighted sum of the coordinates of the four virtual control points; obtaining the relationship between each vertex of the marker and the four virtual control points in the camera coordinate system based on the invariance of linear relationships under Euclidean transformation; calculating the relationship between each virtual control point and an image point based on the image coordinates corresponding to the vertex coordinates of the marker in the workpiece under test, the weighted sum of the coordinates of the four virtual control points in the camera coordinate system, and the camera intrinsic parameters; calculating the distance between the four virtual control points and the coordinates of the four virtual control points in the camera coordinate system; and calculating the pose and distance information of the marker in the workpiece under test to the camera based on the coordinates of the four virtual control points in the camera coordinate system.

8. A device for measuring workpiece distance on an assembly line based on monocular vision, characterized in that, include: A camera used to capture images according to control commands; A mobile device that controls the camera to move according to control commands; Memory, used to store computer programs; A host computer is used to execute the computer program to implement the steps of the monocular vision-based assembly line workpiece distance measurement method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the monocular vision-based assembly line workpiece distance measurement method as described in any one of claims 1 to 6.

Images