A workpiece feature measurement method based on 3D machine vision local field of view

Abstract

The present invention discloses a workpiece feature measurement method based on 3D machine vision local field of view, which belongs to the field of computer vision measurement. First, a single local field of view image of the target workpiece is obtained and preprocessed; then a HED edge detection model and Hough detection are established; the detected pixel coordinates and edges are placed on a proposed platform of the camera field of view, and the coordinate difference between the camera and the pixel point is solved according to the camera working distance through the pixel and real distance scale, and the space vector of the pixel coordinate is obtained, and the conversion from the image pixel coordinate to the basic space vector is completed, and the obtained pixel point space vector is rotated in three dimensions through a rotation system. The present invention solves the problem that the target workpiece feature cannot be measured with high precision under the conditions of limited shooting environment and less image data, and greatly improves the work efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer vision measurement, and in particular to a workpiece feature measurement method based on a local field of view of 3D machine vision. Background Art

[0002] In practical problems, the method of measuring the characteristic parameters of workpieces through images is often limited by objective factors such as equipment performance, shooting environment and number of images, which may lead to poor recognition effect. In addition, at the technical level, mapping the pixel features in the captured image to spatial coordinates is also a relatively complex problem, involving knowledge and models such as pixel-to-vector conversion and three-dimensional reconstruction. Improper handling will also affect the measurement accuracy of the target workpiece. Therefore, combining computer vision technology to measure the global position and physical characteristics of planar workpieces through a single local field of view image is the main innovative content of the present invention. Effective measurement and calibration of workpiece position and other physical characteristic parameters are achieved under the constraints of the objective environment. This method can not only improve the efficiency and accuracy of the production process, but also reduce human measurement errors, thereby promoting the intelligent and automated development of industrial production. Summary of the invention

[0003] The object of the present invention is to provide a method for measuring planar workpiece features through a single local field of view image based on 3D machine vision, so as to solve various problems arising in the target workpiece measurement process proposed in the above background technology.

[0004] To achieve the above object, the present invention adopts the following technical solution: a workpiece feature measurement method based on 3D machine vision local field of view, comprising the following steps:

[0005] Step 1: Obtain a single local field of view image of the target workpiece;

[0006] Step2: Preprocess the image obtained in Step1;

[0007] Step 3: Based on the preprocessed image obtained in Step 2, establish the HED edge detection model and Hough detection;

[0008] Step 4: Place the pixel coordinates and edges detected in Step 3 on the proposed platform of the camera's field of view, and use the pixel and real distance scale to solve the coordinate difference between the camera and the pixel point according to the camera working distance, and obtain the spatial vector of the pixel coordinates, thus completing the conversion from the image pixel coordinates to the basic spatial vector;

[0009] Step 5: The spatial vector of the pixel point obtained in Step 4 is rotated three-dimensionally by a rotation system, and the intersection with the zero plane is the plane coordinate corresponding to the pixel point, that is, the plane coordinate position of all edge points of the local field of view image is obtained;

[0010] Step 6: Apply the least squares method to fit the local edge points in Step 5 to solve the unknown data points, and then calculate and measure the physical feature information of the complete target workpiece.

[0011] Furthermore, the method for obtaining a single local field of view image of the target workpiece in Step 1 is: fixing the camera on the track so that the camera field of view is vertical, photographing a partial position of the target object, and obtaining a local field of view image obtained when the camera field of view is limited.

[0012] Furthermore, in Step 2, the image obtained in Step 1 is rotated to the workbench viewing angle to increase the brightness of the original image, and the three-channel image is converted into a grayscale image, and then binarized and expanded for preprocessing.

[0013] Furthermore, in Step 3, the specific steps of establishing the HED edge detection model and Hough detection are as follows:

[0014] a. Based on DCNN, ResNe is selected as the pre-training model to output edge feature maps of multiple scales;

[0015] b. Calculate the gradient size and direction of each pixel in the image, use non-maximum suppression and dual threshold detection to determine the real and potential edges, and finally suppress isolated weak edges to complete Hough detection;

[0016] c. The target edge is determined by the HED edge detection and Hough detection results;

[0017] Furthermore, the method of establishing the HED edge detection model is: select a pre-trained model, replace the fully connected layer of the network with a convolutional layer to form a holistically-nested structure, calculate the loss function based on the difference between the annotated edge image and the output feature map, and feed it back to update the network parameters. The trained network model can be used for edge detection.

[0018] Furthermore, the method of converting pixel coordinates to space vectors in Step 4 is as follows: a proposed platform is constructed based on the camera field of view, the projection point of the camera on the platform is located at the center, and the obtained pixel points are placed on the platform. The scale between the pixel distance measured by the system and the real distance is used to calculate the mapping position of the pixel coordinates and the space coordinates, and the spatial vector of the pixel point is finally obtained by calculating the camera working distance and the spatial coordinates of the camera. This method is applied to all edge pixels to complete the conversion from image pixel coordinates to basic space vectors.

[0019] Beneficial effects of the present invention: The present invention performs grayscale conversion, binarization and expansion operations on a single local field of view image of a workpiece to enhance the edge pixels of the image; the image edge is determined by HED edge detection and Hough detection to ensure the accuracy of edge data; the spatial vector of the extracted pixel points and the plane coordinates corresponding to the workpiece are determined based on the establishment of a proposed platform and HPB rotation; the local field of view data points are fitted using the least squares method to complete the calculation and measurement of the required features of the global workpiece and ensure that the error is relatively minimal. The present invention solves the problem that the target workpiece features cannot be measured with high precision under conditions such as limited shooting environment and less image data, and greatly improves work efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the shooting environment;

[0021] Figure 2 A schematic diagram of the proposed platform is established based on the camera's field of view. DETAILED DESCRIPTION

[0022] The following is a detailed description of a workpiece feature measurement method based on a local field of view of 3D machine vision according to the present invention with reference to examples and drawings.

[0023] The shooting environment described in the example is as follows Figure 1 As shown in the figure, the camera position is selected and fixed on the track to ensure that the camera field of view is vertical to prevent the measurement from being affected by the angle tilt. Part of the target object is photographed to simulate the local field of view image obtained when the camera field of view is limited.

[0024] After obtaining a single partial field of view image of the workpiece, rotate it to the workbench perspective. Since the origin of the basic coordinate system is located in the center, and the index origin of the image is located in the upper left corner, it is necessary to perform a translation process to move its origin to the center of the coordinate system. Then increase the brightness of the original image, convert it into a grayscale image, perform binarization and dilation, and convolve it with the original partial field of view image to increase the pixel intensity at the edge of the image, expand its white area, that is, the target object area photographed, and obtain the preprocessed partial field of view image.

[0025] Then, the HED edge detection model is established, and the local field of view image is smoothed using a Gaussian filter to eliminate noise interference. Based on DCNN, ResNet is selected as the pre-training model, and the fully connected layer of ResNet is replaced with a convolutional layer to form a holistically-nested structure. Multi-scale convolutional layers are introduced into the structure to capture image edge features at different levels, so that it can simultaneously output multiple edge feature maps of different scales to obtain more accurate edge detection results. At the same time, Hough detection is applied to calculate the gradient size and direction of each pixel in the local field of view image, and non-maximum suppression is used to eliminate the adverse effects of edge detection. At the same time, real and potential edges are determined through dual threshold detection, and finally isolated weak edges are suppressed to complete edge detection. With the help of HED edge detection results, the target edge is determined by the two models to ensure the accuracy of the extracted edge, providing high-quality data for the subsequent measurement of workpiece size based on edge pixel coordinates.

[0026] Then, a proposed platform is established based on the camera field of view, and the pixel coordinates are mapped to the geometric space to facilitate the accurate measurement and analysis of the workpiece features, such as Figure 2 As shown: The platform projection of the camera is located at the center. By fixing the focal length of the lens, the actual field of view width and height of the camera can be calculated as follows:

[0027] In the formula, FW is the field of view width, FH is the field of view height, WD is the working distance, HF is the horizontal viewing angle, VF is the vertical viewing angle. Let the local field of view image overlap with the camera field of view, and construct the actual distance and pixel distance scale. Convert the edge pixel coordinate position to the actual space coordinate, consider the camera working distance, and calculate the coordinate difference, which is the space vector corresponding to the pixel point. After applying it to the coordinates of all edge pixels, all space vectors can be obtained.

[0028] Furthermore, the three-dimensional space vector is converted into two-dimensional coordinates to measure the actual situation of the target workpiece. The three-dimensional vector represents the position of the workpiece, and the rotation matrix represents the posture. Therefore, the position and posture of the workpiece in space can be represented by a 4x4 homogeneous transformation matrix: ,in R is a 3×3 rotation matrix belonging to a special orthogonal group , t is a 3D translation vector, also belonging to the real number space R 3. The lower right corner element of the matrix is ​​1, indicating the scaling factor in homogeneous coordinates. The lower left corner of the matrix is ​​the zero vector to ensure the homogeneous coordinate form. Therefore, considering the homogeneous coordinate representation, the obtained three-dimensional vector is expanded to four dimensions, and the expanded element is set to 1 to ensure the simplicity of the translation transformation. The improved vector representation of the edge pixel point is as follows: , and then the HPB rotation system is used to rotate the 3D reconstruction to calculate the corresponding vectors of the image feature points. The rotation relationship between the HPB rotation system and the XYZ axis is shown in Table 1:

[0029] Table 1. Correspondence between HPB coordinate system and three axes

[0030]

[0031] Rotate a set of vectors in the space vector along the XYZ axis and calculate it through the rotation matrix identity relationship. Suppose the space vector before rotation is , the rotation angles around each axis are:

[0032] Rotation along the X axis: ; Rotate along the Y axis: , rotate along the Z axis: , and then the vector of the edge point after rotation is obtained. The intersection of each vector and the Z=0 plane is the plane coordinate corresponding to the pixel point, realizing the conversion of edge pixel coordinates to space plane coordinates.

[0033] Finally, the least squares method is applied to minimize the sum of square errors between the predicted data and the real data, thereby obtaining the edge fitting curve of the local image. According to the geometric relationship between the circle parameters, the coordinates of the center of the circle are determined by the arc length and the vertical line, and the distance between the edge point and the center of the circle is the radius, thereby calculating and measuring the complete edge, center and radius features or data of the target workpiece.

[0034] The present invention aims to solve the problem that traditional manual measurement has low accuracy or even cannot be effectively measured when the shooting scene is limited. By capturing local feature information through a single local field of view image, the present invention combines 3D machine vision, spatial mapping and rotation model to achieve analysis and measurement of the global position and physical characteristics of the workpiece, solving the high-precision measurement problem faced by traditional measurement technology when acquiring global data.

[0035] The above-mentioned examples and drawings have been used to describe the implementation method of the present invention in detail, but the present invention is not limited to the above-mentioned implementation methods. Any changes made by researchers familiar with the technical field within the scope disclosed by the patent of the present invention without departing from the purpose of the present invention are all within the scope of protection of the patent of the present invention.

Claims

1. A workpiece feature measurement method based on 3D machine vision local field of view, characterized in that: The following steps are involved: Step 1: Obtain a single local field of view image of the target workpiece; Step2: Preprocess the image obtained in Step1; Step 3: Based on the preprocessed image obtained in Step 2, establish the HED edge detection model and Hough detection; The specific steps are as follows: a. Based on DCNN, ResNe is selected as the pre-training model to output edge feature maps of multiple scales; b. Calculate the gradient size and direction of each pixel in the image, use non-maximum suppression and dual threshold detection to determine the real and potential edges, and finally suppress isolated weak edges to complete Hough detection; c. The target edge is determined by the HED edge detection and Hough detection results; The method of establishing the HED edge detection model is as follows: select a pre-trained model, replace the fully connected layer of the network with a convolutional layer to form a holistically-nested structure, calculate the loss function by the difference between the annotated edge image and the output feature map, and feed it back to update the network parameters. The trained network model can be used for edge detection. Step 4: Place the pixel coordinates and edges detected in Step 3 on the proposed platform of the camera's field of view, and use the pixel and real distance scale to solve the coordinate difference between the camera and the pixel point according to the camera working distance, and obtain the spatial vector of the pixel coordinates, thus completing the conversion from the image pixel coordinates to the basic spatial vector; Step 5: The spatial vector of the pixel point obtained in Step 4 is rotated three-dimensionally by a rotation system, and the intersection with the zero plane is the plane coordinate corresponding to the pixel point, that is, the plane coordinate position of all edge points of the local field of view image is obtained; Step 6: Apply the least squares method to fit the local edge points in Step 5 to solve the unknown data points, and then calculate and measure the physical feature information of the complete target workpiece.

2. A workpiece feature measurement method based on 3D machine vision local field of view according to claim 1, characterized in that: The method for obtaining a single local field of view image of the target workpiece in Step 1 is: fixing the camera on the track so that the camera field of view is vertical, photographing a part of the target object, and obtaining a local field of view image obtained when the camera field of view is limited.

3. The workpiece feature measurement method based on 3D machine vision local field of view according to claim 1, characterized in that: In Step 2, the image obtained in Step 1 is rotated to the workbench perspective to increase the brightness of the original image. At the same time, the three-channel image is converted into a grayscale image, and then binarized and expanded for preprocessing.

4. The workpiece feature measurement method based on 3D machine vision local field of view according to claim 1, characterized in that: Step 4: The method of converting pixel coordinates to spatial vectors is as follows: a simulated platform is constructed based on the camera field of view, the projection point of the camera on the platform is located at the center, and the obtained pixel points are placed on the platform. The scale between the pixel distance measured by the system and the real distance is used to calculate the mapping position of the pixel coordinates and the spatial coordinates. The camera working distance and the camera's spatial coordinates are used to perform operations, and finally the spatial vector of the pixel point is obtained. This method is applied to all edge pixel points to complete the conversion from image pixel coordinates to basic spatial vectors.

Images