A method for implementing three-dimensional coordinate measurement based on machine vision

Abstract

The application designs a three-dimensional coordinate measurement method based on machine vision, which comprises the following steps: an improved yolov5 model is used to realize real-time detection and recognition of the measured object; an algorithm for pre-processing the real-time picture of the binocular camera; an algorithm for using improved AD-Census to perform stereo matching on the processed image; and an algorithm for generating the three-dimensional coordinates of the recognized object according to the depth map. The application solves the measurement of the three-dimensional coordinates of the recognized object, uses the improved yolov5 algorithm to generate the position of the recognized object in the image, and then maps the position to the depth map obtained through stereo matching, so that the three-dimensional coordinates of the recognized object can be obtained. The method improves the precision of measuring the three-dimensional coordinates, and lays a foundation for unmanned driving, automatic obstacle avoidance technology and use of a robot arm to grasp an object.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, in particular to a method for realizing three-dimensional coordinate measurement based on a machine vision system. BACKGROUND

[0002] With the continuous development of industrial technology, the theory of artificial intelligence and related technology research has developed rapidly. The proposal of "Intelligent Manufacturing 2025" marks that industrial production is gradually moving towards intelligentization, and the application proportion of artificial intelligence in various fields is increasing year by year. Machine vision is one of the important branches of artificial intelligence, so machine vision is also a hot research topic at present.

[0003] With the continuous research in the field of robot technology, as the "eyes" of the robot, machine vision is widely used in object recognition, classification and other fields. According to statistics, 80% of the information obtained by humans from the outside world is through vision. Through vision, the color, size, shape and position of the object can be obtained. In order to enable the robot to avoid obstacles autonomously through vision, depth information is the most important research.

[0004] Therefore, a three-dimensional coordinate measurement method based on a machine vision system is needed to realize the above functions, and to autonomously identify objects and measure the three-dimensional coordinate information of the objects in a set environment, providing protection for unmanned driving and robot autonomous obstacle avoidance technology. SUMMARY

[0005] The purpose of the present application is to propose an implementation method of three-dimensional coordinate measurement based on a machine vision system, which first uses an improved Gaussian algorithm for image denoising processing on the real-time picture taken by a binocular camera, and improves the sigma parameter in the Gaussian function, so that when the value of a target pixel point and a pixel point in the neighborhood is greatly different, the weight matrix in this direction is larger, so as to improve the image quality; then the processed image is matched with the cost, and the gray image channel is added on the basis of the original, and the Hamming distance calculated by the average value of the original red, green and blue three channels becomes the average value of the present red, green, blue and gray four channels, and the gray image is also introduced in the matching cost aggregation work, the gradient value in the neighborhood of the target pixel point is calculated, the adaptive window is generated according to the set rule, and then the depth map is obtained through disparity optimization and post-processing, the position mapping of the depth map and the left eye image is carried out, the CSP2_X structure in the Neck part of the yolov5 algorithm is improved, the output of each CBL layer in the CSP2_X structure is processed through the Conv layer, and the processing result is sent into the Concat layer, the accuracy of object recognition is improved through the improvement of the yolov5 algorithm, and finally the position of the recognized object is mapped to the depth map, and the three-dimensional coordinate information of the center point of the object is calculated through the triangular principle; it is believed that the present application can be applied in many fields, and provides the possibility for the unmanned driving technology and robot obstacle avoidance. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 The system scheme of the present application is shown in the figure;

[0007] Figure 2 The software flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0008] The present application relates to an implementation method of three-dimensional coordinate measurement based on a machine vision system, which is designed to be intelligent and convenient at the beginning, and can run in window and linux systems, and can also be imported into FPGA and jetsonnano development board for running, and has strong portability;

[0009] The specific implementation method is as follows:

[0010] (1) First, configure the python software and python library required by the present application in the operating system;

[0011] (2) Connect the binocular camera to the operating system and calibrate it;

[0012] (3) Write the calibration data into the calibration function of the algorithm;

[0013] (4) Use the improved yolov5 algorithm to train the recognized object, and save the optimal weight;

[0014] (5) Introduce the optimal weight into the algorithm;

[0015] (6) Click the run button and view the returned three-dimensional coordinates in the python terminal.

[0016] As an embodiment of the present application, a method for implementing three-dimensional coordinate measurement based on machine vision, the method comprising the following steps:

[0017] (1) Real-time detection and recognition of the measured object is realized by the improved yolov5 model algorithm;

[0018] (2) The algorithm for pre-processing the real-time image of the binocular camera;

[0019] (3) The algorithm for using improved AD-Census for stereo matching of the processed image;

[0020] (4) The algorithm for generating three-dimensional coordinates of the recognized object according to the depth map.

[0021] As an embodiment of the present application, in step (1), the CSP2_X structure in the Neck part of the yolov5 algorithm is improved, and the output of each CBL layer in the CSP2_X structure is processed by the Conv layer, and the processing result is sent to the Concat layer, so as to improve the recognition accuracy of the grabbed object. After the above steps, the target detection training is performed on the grabbed object, and the optimal weight in the training process is obtained, and then the model is deployed for real-time recognition.

[0022] As an embodiment of the present application, in step (2), the algorithm is deployed on the development board of jetson nano, which first acquires the real-time image collected by the binocular camera, and then uses the improved Gaussian filter algorithm for image noise reduction processing frame by frame, and improves the sigma parameter in the Gaussian function. When the value of the target pixel point and the value of the pixel point in the neighborhood are greatly different, the weight matrix in this direction is larger, so as to filter the salt and pepper noise generated when the camera shoots.

[0023] As an embodiment of the present application, in step (3), the processed image is sequentially subjected to matching cost calculation, matching cost aggregation, disparity optimization and post-processing steps. In the matching cost calculation, the gray image channel is added, and the calculated Hamming distance is changed from the original red, green and blue three-channel average to the current red, green, blue and gray four-channel average. In the matching cost aggregation, the gray image is also added, and the gradient value in the neighborhood of the target pixel point is calculated. The smaller the gradient value, the larger the adaptive window range but not exceeding the maximum set value. Through the algorithm, the depth map of the camera shot can be generated.

[0024] As an embodiment of the present application, in step (4), the position of the measured object in the image is identified according to the improved yolov5 algorithm, the generated depth map is mapped with the original image captured by the left eye camera, the corresponding position is obtained, the midpoint of the identified object is selected as the target point for measuring the three-dimensional coordinates, the pixel position coordinates corresponding to the point in the left and right images are obtained, and the three-dimensional coordinate information is calculated according to the triangular principle.

Claims

1. A method for implementing three-dimensional coordinate measurement based on machine vision, characterized in that, The method includes the following steps: (1) An algorithm for real-time detection and recognition of the object under test is implemented using an improved yolov5 model; (2) Algorithm for preprocessing real-time images from binocular cameras; (3) Use the improved AD-Census algorithm for stereo matching on the processed image; (4) An algorithm for generating the three-dimensional coordinates of the identified object based on the depth map; In step (1), the CSP2_X structure in the Neck part of the YOLOv5 algorithm is improved. The output of each CBL layer in the CSP2_X structure is processed by the Conv layer, and the processing result is sent to the Concat layer to improve the recognition accuracy of the grabbed item. After the above steps, the grabbed item is trained for target detection to obtain the optimal weight during the training process, and then the model is deployed for real-time recognition. In step (3), the processed image is sequentially subjected to matching cost calculation, matching cost aggregation, disparity optimization and post-processing steps. In the matching cost calculation, grayscale image channels are added, and the calculated Hamming distance changes from the original average of the three red, green and blue channels to the current average of the four red, green, blue and gray channels. In the matching cost aggregation, grayscale image is also added, and the gradient value in the neighborhood of the target pixel is calculated. The smaller the gradient value, the larger the range of the adaptive window, but it does not exceed the maximum set value. The depth map captured by the camera can be generated through this algorithm.

2. The method for implementing three-dimensional coordinate measurement based on machine vision according to claim 1, characterized in that: In step (2), the algorithm is deployed on the Jetson Nano development board. The algorithm first acquires the real-time images captured by the binocular camera, and then performs image noise reduction processing on each frame of the image using an improved Gaussian filtering algorithm. The σ parameter in the Gaussian function is improved. When the value of the target pixel is much different from that of a pixel in the neighborhood, the weight matrix assigned in that direction is larger, so as to filter the salt-and-pepper noise generated when the camera is shooting.

3. The method for implementing three-dimensional coordinate measurement based on machine vision according to claim 1, characterized in that: In step (4), the position of the object being measured in the image is identified according to the improved YOLOv5 algorithm. The generated depth map is mapped with the original image captured by the left eye camera to obtain the corresponding position. The midpoint of the object being identified is selected as the target point for measuring the three-dimensional coordinates. The three-dimensional coordinate information can be calculated based on the pixel position coordinates of the corresponding point in the left and right images according to the trigonometric principle.

Images