Profile section size measurement method and system based on two-dimensional image and three-dimensional point cloud

Abstract

The invention discloses a sectional material section dimension measurement method and system based on a two-dimensional image and a three-dimensional point cloud, and relates to the technical field of sectional material detection, and the method comprises the steps: obtaining a two-dimensional image and a depth image of a sectional material section to be measured, and generating three-dimensional point cloud data according to the depth image; respectively preprocessing the two-dimensional image and the three-dimensional point cloud data to correspondingly obtain the preprocessed two-dimensional image and the preprocessed three-dimensional point cloud data, and inputting the preprocessed two-dimensional image and the preprocessed three-dimensional point cloud data into the trained burr detection model to obtain a burr region in the two-dimensional image; removing the burr area on the two-dimensional image to obtain a burr-free image of the section of the profile to be measured; and correcting the burr-free image of the section of the to-be-measured section to obtain a corrected image of the section of the to-be-measured section, and measuring size parameters on the corrected image of the section of the to-be-measured section. According to the method, the real deburring process treatment operation can be replaced, the time cost for waiting for manual polishing is shortened, errors in the imaging and projection process are eliminated, it is ensured that size parameters are measured on an interference-free image, and the detection accuracy is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of profile detection, and more specifically, to a profile cross-sectional dimension measurement method and system based on two-dimensional images and three-dimensional point clouds. Background Art

[0002] In the production process of profiles, materials are formed into specific cross-sectional shapes through extrusion and cutting processes. In order to ensure that the dimensions of these sections meet the design specifications and to ensure that the extrusion die does not wear and tear that affects product quality, accurate measurement of the profile section is a crucial step. In actual production, the profile section obtained after cutting is often accompanied by burrs, which significantly interfere with the measurement of the section size.

[0003] In the prior art, the method for measuring the cross-sectional dimensions of profiles is usually to manually polish and remove burrs before measuring. This method has the following problems: the manual polishing process is time-consuming and labor-intensive, the labor cost is high, and it is difficult to meet the needs of modern industrial production. Manual measurement relies on the operator's experience and judgment, which is highly subjective and can easily lead to inconsistent measurement results due to individual differences or misjudgment, reducing the reliability of detection. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method for measuring the cross-sectional dimensions of a profile based on two-dimensional images and three-dimensional point clouds, so as to overcome the above-mentioned defect that the cross-sectional dimensions of a profile cannot be accurately measured due to the influence of burrs in industrial production; the second purpose is to provide a system for measuring the cross-sectional dimensions of a profile based on two-dimensional images and three-dimensional point clouds.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] The present invention provides a method for measuring the cross-sectional dimensions of a profile based on a two-dimensional image and a three-dimensional point cloud, comprising:

[0007] Acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image;

[0008] Preprocessing the two-dimensional image and the three-dimensional point cloud data respectively to obtain the preprocessed two-dimensional image and three-dimensional point cloud data respectively;

[0009] Inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image;

[0010] Removing the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured;

[0011] The burr-free image of the cross section of the profile to be measured is corrected to obtain a corrected image of the cross section of the profile to be measured, and the dimensional parameters are measured on the corrected image of the cross section of the profile to be measured.

[0012] Preferably, the step of acquiring a two-dimensional image and a depth image of the cross section of the profile to be measured comprises:

[0013] The profile section to be measured is fixed, a two-dimensional image of the profile section to be measured is obtained by using a two-dimensional industrial camera, a depth image of the profile section to be measured is obtained by using a three-dimensional depth camera, and the position and posture information of the three-dimensional depth camera and the two-dimensional industrial camera are saved.

[0014] Preferably, generating three-dimensional point cloud data according to the depth image includes:

[0015] The formula for generating 3D point cloud data from a depth image is:

[0016]

[0017] Where, X i , Y i , Z i Represents the 3D coordinates of the i-th point cloud in the 3D point cloud data, u i , v i Represents the two-dimensional coordinates of the i-th pixel in the depth image, depth(u i , v i ) represents the depth value of the i-th pixel, c x , c y represents the optical center coordinates of the 3D depth camera, f x , f y Respectively represent the focal length of the 3D depth camera in the x and y directions.

[0018] Preferably, the posture information of the 3D depth camera includes a 3D camera rotation matrix and a 3D camera translation vector;

[0019] According to the relative position relationship between the 3D depth camera and the 2D industrial camera, the pose information of the 2D industrial camera is calculated, including the 2D camera rotation matrix and the 2D camera translation vector; the calculation formula is:

[0020] R 2D =R rel ·R 3D

[0021] t 2D =R rel ·t 3D +t rel

[0022] In the formula, R 3D , t 3DRepresents the 3D camera rotation matrix and 3D camera translation vector, R rel , t rel Represents the relative position rotation matrix and relative position translation vector of the 3D depth camera and the 2D industrial camera, R 2D , t 2D Represent the 2D camera rotation matrix and 2D camera translation vector respectively.

[0023] Preferably, preprocessing the two-dimensional image and the three-dimensional point cloud data respectively to obtain the preprocessed two-dimensional image and three-dimensional point cloud data respectively includes:

[0024] The two-dimensional image is sequentially subjected to grayscale processing, denoising processing, adaptive threshold binarization processing and connected domain analysis to obtain a preprocessed two-dimensional image;

[0025] The three-dimensional point cloud data is subjected to denoising processing to obtain pre-processed three-dimensional point cloud data.

[0026] Preferably, the calculation formula for the grayscale processing is specifically:

[0027]

[0028] Among them, I(x, y) represents the grayscale pixel value of the pixel at the coordinate (x, y), R(x, y), G(x, y), and B(x, y) respectively represent the red, green, and blue components of the pixel at the coordinate (x, y) in the two-dimensional image.

[0029] Preferably, a median filter algorithm is used for denoising, and the calculation formula of the median filter is:

[0030] I′(x,y)=median(I(x+i,y+j)),-k≤i,j≤k

[0031] Where I′(x, y) represents the pixel value at the coordinate (x, y) after denoising, I(x+i, y+j) represents the pixel value in the (2k+1)×(2k+1) window centered on the pixel at (x, y) in the grayscale two-dimensional image, and median represents the median of all pixel values ​​in the window.

[0032] It is understandable that there will be inevitable noise in the image acquisition process, such as that caused by uneven lighting; in order to remove this noise while retaining the edge information in the image, the median filtering algorithm is used for denoising. The principle of median filtering is to replace the value of each pixel in the image with the median of its neighboring pixels, thereby effectively removing isolated noise points without blurring the edges.

[0033] Preferably, the adaptive threshold binarization process includes:

[0034] Calculate the adaptive threshold:

[0035]

[0036] In the formula, T(x, y) represents the adaptive threshold of the pixel at the coordinate (x, y), ω represents the size of the binary window, and C represents a constant used to adjust the degree of separation between the foreground and the background;

[0037] Binarization is performed based on the adaptive threshold:

[0038]

[0039] Where I_b(x, y) represents the pixel value after binarization at the coordinate (x, y), 0 represents the background pixel, and 255 represents the foreground pixel.

[0040] Preferably, the connected domain analysis includes:

[0041] Use the breadth-first search algorithm to mark the connected domains of the two-dimensional image after binarization and identify all connected domains;

[0042] Calculate the number of pixels in each connected domain, and calculate the area of ​​each connected domain based on the number of pixels and pixel values;

[0043] The connected domain with the largest area is retained, and the remaining connected domains are set as the background to obtain the preprocessed two-dimensional image.

[0044] Preferably, the calculation formula of the area of ​​the connected domain is specifically:

[0045]

[0046] Among them, A i Represents the area of ​​the i-th connected domain, Region i Represents the coordinate set of all foreground pixels in the i-th connected component.

[0047] Preferably, performing denoising processing on the three-dimensional point cloud data to obtain pre-processed three-dimensional point cloud data includes:

[0048] For each point in the three-dimensional point cloud data, searching for a set of adjacent points within a preset radius;

[0049] Calculate the mean distance between the point and its adjacent point set. The calculation formula is:

[0050]

[0051] Where, d mean (P i ) represents the mean distance between the i-th point and its adjacent points, Ni represents the adjacent point set of the i-th point, P j represents the jth point in the adjacent point set, ||*|| represents the Euclidean distance;

[0052] Traversing each point in the three-dimensional point cloud data to obtain the mean distance between each point and its adjacent point set;

[0053] Calculate the mean and standard deviation of the global point cloud according to the mean distance between each point and its adjacent point set;

[0054] The threshold is calculated according to the mean and standard deviation of the global point cloud, and the calculation formula is:

[0055] τ=μ+α·f

[0056] In the formula, τ represents the threshold, μ represents the mean of the global point cloud, f represents the standard deviation of the global point cloud, and f represents the scale factor;

[0057] Compare the mean distance between each point and its adjacent point set and the threshold in turn. If the mean distance between the point and its adjacent point set is greater than the threshold, the point is removed as an outlier; otherwise, the point is retained.

[0058] Preferably, obtaining the trained burr detection model includes:

[0059] Obtaining 3D point cloud data and corresponding 2D images of several profile sections;

[0060] Normalizing the three-dimensional point cloud data and the corresponding two-dimensional images of all the profile sections to obtain the normalized three-dimensional point cloud data and the corresponding two-dimensional images of the profile sections;

[0061] Labeling each point in the normalized three-dimensional point cloud data of all the profile sections into a category to obtain three-dimensional point cloud data of the profile sections with category labels;

[0062] Construct a burr detection model, input the three-dimensional point cloud data with category labels and the corresponding two-dimensional image of the profile section, set the loss function for optimization training, and when the loss function value reaches the minimum, save the corresponding model parameters to obtain the trained burr detection model.

[0063] Preferably, the burr detection model includes a two-dimensional feature extraction unit, a first multi-layer perceptron, a first three-dimensional feature down-sampling unit, a first feature fusion unit, a second three-dimensional feature down-sampling unit, a second feature fusion unit, a third three-dimensional feature down-sampling unit, a third feature fusion unit, a fourth three-dimensional feature down-sampling unit, a fourth feature fusion unit, a second multi-layer perceptron, a first three-dimensional feature up-sampling unit, a second three-dimensional feature up-sampling unit, a third three-dimensional feature up-sampling unit, a fourth three-dimensional feature up-sampling unit and a third multi-layer perceptron;

[0064] The output end of the two-dimensional feature extraction unit is connected to the first input end of the first feature fusion unit, the second feature fusion unit, the third feature fusion unit, and the fourth feature fusion unit respectively;

[0065] The output end of the first multi-layer perceptron is connected to the input end of the first three-dimensional feature downsampling unit, the output end of the first three-dimensional feature downsampling unit is connected to the second input end of the first feature fusion unit, the output end of the first feature fusion unit is connected to the input end of the second three-dimensional feature downsampling unit, the output end of the second three-dimensional feature downsampling unit is connected to the second input end of the second feature fusion unit, the output end of the second feature fusion unit is connected to the input end of the third three-dimensional feature downsampling unit, the output end of the third three-dimensional feature downsampling unit is connected to the second input end of the third feature fusion unit, the output end of the third feature fusion unit is connected to the input end of the fourth three-dimensional feature downsampling unit, the output end of the fourth three-dimensional feature downsampling unit is connected to the second input end of the fourth feature fusion unit, and the output end of the fourth feature fusion unit is connected to the input end of the second multi-layer perceptron;

[0066] The output end of the second multilayer perceptron is connected to the first input end of the first three-dimensional feature upsampling unit, the output end of the first three-dimensional feature upsampling unit is connected to the first input end of the second three-dimensional feature upsampling unit, the output end of the second three-dimensional feature upsampling unit is connected to the first input end of the third three-dimensional feature upsampling unit, the output end of the third three-dimensional feature upsampling unit is connected to the first input end of the fourth three-dimensional feature upsampling unit, and the output end of the fourth three-dimensional feature upsampling unit is connected to the input end of the third multilayer perceptron;

[0067] The output end of the first feature fusion unit is also connected to the second input end of the fourth three-dimensional feature upsampling unit, the output end of the second feature fusion unit is also connected to the second input end of the third three-dimensional feature upsampling unit, the output end of the third feature fusion unit is also connected to the second input end of the second three-dimensional feature upsampling unit, and the output end of the fourth feature fusion unit is also connected to the second input end of the first three-dimensional feature upsampling unit.

[0068] Preferably, the structures of the first three-dimensional feature downsampling unit, the second three-dimensional feature downsampling unit, the third three-dimensional feature downsampling unit and the fourth three-dimensional feature downsampling unit are the same, and all include a farthest point sampling layer, a K nearest neighbor grouping layer, a neighborhood maximum pooling layer, a first fully connected layer, a first attention layer and a second fully connected layer connected in sequence; the output end of the neighborhood maximum pooling layer is also connected to the output end of the second fully connected layer.

[0069] Preferably, the first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit have the same structure, and all include a first cross attention layer, a first self-attention layer, a second cross attention layer, a second self-attention layer and a third cross attention layer connected in sequence;

[0070] The output end of the two-dimensional feature extraction unit is connected to the first input end of the first cross-attention layer and the second cross-attention layer respectively, and the output end of the second fully connected layer is connected to the second input end of the first cross-attention layer; the output end of the first self-attention layer is also connected to the first input end of the third cross-attention layer.

[0071] Preferably, the first three-dimensional feature upsampling unit, the second three-dimensional feature upsampling unit, the third three-dimensional feature upsampling unit and the fourth three-dimensional feature upsampling unit have the same structure, and all include a third fully connected layer, a fourth fully connected layer, a trilinear interpolation layer, an element-by-element addition layer, a fifth fully connected layer, a third self-attention layer and a sixth fully connected layer;

[0072] The fourth fully connected layer, the trilinear interpolation layer, the element-by-element addition layer, the fifth fully connected layer, the third self-attention layer and the sixth fully connected layer are connected in sequence;

[0073] The input end of the third fully connected layer is connected to the output end of the third cross attention layer, the output end of the third fully connected layer is connected to the second input end of the element-by-element addition layer, and the output end of the element-by-element addition layer is also connected to the output end of the sixth fully connected layer.

[0074] Preferably, measuring the dimensional parameters on the calibrated image of the cross section of the profile to be measured includes:

[0075] Performing edge detection on the corrected image of the profile section to be measured using an edge detection algorithm to obtain a set of contour lines of the profile section to be measured;

[0076] Arbitrarily select a preset measurement position, and select two corresponding target contour lines from the contour line set of the profile section to be measured;

[0077] Calculating the pixel distance between two target contour lines in the corrected image of the cross section of the profile to be measured;

[0078] Calculate the actual distance between the two target contour lines according to the preset proportionality coefficient and the pixel distance, and use the actual distance as the actual size of the preset measurement part;

[0079] The preset measuring parts are traversed to obtain the actual sizes of all the preset measuring parts, and the size parameters of the calibrated image of the cross section of the profile to be measured are formed.

[0080] The present invention also provides a profile cross-sectional dimension measurement system based on two-dimensional images and three-dimensional point clouds, which is used to implement the above method, including:

[0081] A data acquisition module, used to acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image;

[0082] A data processing module, used to preprocess the two-dimensional image and the three-dimensional point cloud data respectively, and obtain the preprocessed two-dimensional image and three-dimensional point cloud data accordingly;

[0083] A burr detection module, used for inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image;

[0084] A burr removal module, used to remove the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured;

[0085] The dimension measurement module is used to correct the burr-free image of the cross section of the profile to be measured, obtain the corrected image of the cross section of the profile to be measured, and measure the dimension parameters on the corrected image of the cross section of the profile to be measured.

[0086] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0087] The present invention first obtains a two-dimensional image and a depth image of the cross section of the profile to be measured, and generates three-dimensional point cloud data according to the depth image, and by fusing the plane geometric information of the two-dimensional image and the spatial geometric information of the three-dimensional point cloud, ensures that the features of different dimensions can be fully utilized in the subsequent burr detection process, thereby improving the detection accuracy; then, the two-dimensional image and the three-dimensional point cloud data are preprocessed to remove irrelevant environmental information; the preprocessed two-dimensional image and the three-dimensional point cloud data are input into a trained burr detection model, and burr detection is automatically performed to obtain the burr area in the two-dimensional image, thereby replacing the operation of the actual deburring process, reducing the error caused by the difference in operator experience and subjective judgment, reducing the time cost of manual grinding, and improving the consistency, reliability and efficiency of burr detection; finally, the detected burr area is removed on the two-dimensional image, a burr-free image is generated and corrected, and a corrected image of the cross section of the profile to be measured is obtained, and the size parameters are measured on the corrected image of the cross section of the profile to be measured, thereby eliminating the errors in the imaging and projection processes, ensuring that the size parameters are obtained based on the cross-section contour of the profile without interference, thereby greatly improving the detection accuracy and reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 This is a flow chart of a profile cross-sectional dimension measurement method based on two-dimensional images and three-dimensional point clouds as described in Example 1;

[0089] Figure 2 This is a schematic diagram of the structure of the burr detection model described in Example 2;

[0090] Figure 3 A schematic diagram of the structures of the first three-dimensional feature downsampling unit, the second three-dimensional feature downsampling unit, the third three-dimensional feature downsampling unit and the fourth three-dimensional feature downsampling unit described in Example 2;

[0091] Figure 4 It is a schematic diagram of the structures of the first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit described in Example 2;

[0092] Figure 5 A schematic diagram of the structures of the first three-dimensional feature upsampling unit, the second three-dimensional feature upsampling unit, the third three-dimensional feature upsampling unit and the fourth three-dimensional feature upsampling unit according to Embodiment 2;

[0093] Figure 6 This is a structural schematic diagram of a profile cross-sectional dimension measurement system based on two-dimensional images and three-dimensional point clouds described in Example 3. DETAILED DESCRIPTION

[0094] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0095] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0096] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0097] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0098] Example 1

[0099] This embodiment provides a profile cross-sectional dimension measurement method based on two-dimensional images and three-dimensional point clouds. Figure 1 As shown, including:

[0100] S1: Acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image;

[0101] S2: Preprocessing the two-dimensional image and the three-dimensional point cloud data respectively, and obtaining the preprocessed two-dimensional image and three-dimensional point cloud data respectively;

[0102] S3: inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image;

[0103] S4: removing the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured;

[0104] S5: Correcting the burr-free image of the cross section of the profile to be measured to obtain a corrected image of the cross section of the profile to be measured, and measuring dimensional parameters on the corrected image of the cross section of the profile to be measured.

[0105] In the specific implementation process, this embodiment first obtains a two-dimensional image and a depth image of the cross section of the profile to be measured, and generates three-dimensional point cloud data according to the depth image. By fusing the plane geometric information of the two-dimensional image and the spatial geometric information of the three-dimensional point cloud, it is ensured that the features of different latitudes can be fully utilized in the subsequent burr detection process to improve the detection accuracy; then the two-dimensional image and the three-dimensional point cloud data are preprocessed to remove irrelevant environmental information; the preprocessed two-dimensional image and the three-dimensional point cloud data are input into the trained burr detection model, and the burr detection is automatically performed to obtain the burr area in the two-dimensional image, replacing the operation that is actually processed by the deburring process, reducing the error caused by the difference in operator experience and subjective judgment, reducing the time cost of waiting for manual polishing, and improving the consistency, reliability and efficiency of burr detection; finally, the detected burr area is removed on the two-dimensional image, a burr-free image is generated and corrected, and a corrected image of the cross section of the profile to be measured is obtained, and the size parameters are measured on the corrected image of the cross section of the profile to be measured, eliminating the errors in the imaging and projection processes, and ensuring that the size parameters are obtained based on the profile cross section contour without interference, thereby greatly improving the detection accuracy and reliability.

[0106] Example 2

[0107] This embodiment provides a method for measuring profile cross-sectional dimensions based on a two-dimensional image and a three-dimensional point cloud, comprising:

[0108] S1: Acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image;

[0109] The step of obtaining a two-dimensional image and a depth image of the cross section of the profile to be measured comprises:

[0110] Fixing the cross section of the profile to be measured, using a two-dimensional industrial camera to obtain a two-dimensional image of the cross section of the profile to be measured, using a three-dimensional depth camera to obtain a depth image of the cross section of the profile to be measured, and saving the position and posture information of the three-dimensional depth camera and the two-dimensional industrial camera;

[0111] Among them, the pose information of the 3D depth camera includes the 3D camera rotation matrix and the 3D camera translation vector;

[0112] According to the relative position relationship between the 3D depth camera and the 2D industrial camera, the pose information of the 2D industrial camera is calculated, including the 2D camera rotation matrix and the 2D camera translation vector; the calculation formula is:

[0113] R 2D =R rel ·R 3D

[0114] t 2D =R rel ·t 3D +t rel

[0115] In the formula, R 3D , t 3D Represents the 3D camera rotation matrix and 3D camera translation vector, R rel , t rel Represents the relative position rotation matrix and relative position translation vector of the 3D depth camera and the 2D industrial camera, R 2D , t 2D Represent the 2D camera rotation matrix and 2D camera translation vector respectively.

[0116] The formula for generating 3D point cloud data from a depth image is:

[0117]

[0118] Where, X i , Y i , Z i Represents the 3D coordinates of the i-th point cloud in the 3D point cloud data, u i , v i Represents the two-dimensional coordinates of the i-th pixel in the depth image, depth(u i , v i ) represents the depth value of the i-th pixel, c x , c y represents the optical center coordinates of the 3D depth camera, f x , f y Respectively represent the focal length of the 3D depth camera in the x and y directions.

[0119] It can be understood that the depth image is captured by a three-dimensional depth camera, such as a lidar or a structured light camera, where each pixel contains the depth value of the point to the camera. By combining the depth value with the intrinsic and extrinsic parameters of the camera, the depth image can be converted into three-dimensional point cloud data. Through this conversion process, each pixel in the two-dimensional depth image can be mapped to a point in three-dimensional space, thereby generating three-dimensional point cloud data and obtaining the position information of the camera in three-dimensional space: the rotation matrix R 3D and the translation vector t 3D .

[0120] The pose information estimation of a 2D industrial camera is usually based on the pose information of a 3D depth camera, and combined with the relative position relationship between the 3D depth camera and the 2D industrial camera, including the relative position rotation matrix and the relative position translation vector, to derive the pose information of the 2D industrial camera.

[0121] S2: Preprocessing the two-dimensional image and the three-dimensional point cloud data respectively, and obtaining the preprocessed two-dimensional image and three-dimensional point cloud data respectively;

[0122] The two-dimensional image is subjected to grayscale processing, denoising processing, adaptive threshold binarization processing and connected domain analysis in sequence to obtain a preprocessed two-dimensional image;

[0123] S21: Grayscale processing: The calculation formula of the grayscale processing is specifically:

[0124]

[0125] Where I(x, y) represents the grayscale pixel value of the pixel at the coordinate (x, y), R(x, y), G(x, y), and B(x, y) represent the red, green, and blue components of the pixel at the coordinate (x, y) in the two-dimensional image, respectively.

[0126] It can be understood that a two-dimensional image is usually a color image, including three color channels (R, G, B). In order to simplify calculation, the color image needs to be converted into a grayscale image first.

[0127] S22: De-noising: The median filter algorithm is used for denoising. The calculation formula of the median filter is:

[0128] I′(x,y)=median(I(x+i,y+j)),-k≤i,j≤k

[0129] Where I′(x, y) represents the pixel value at the coordinate (x, y) after denoising, I(x+i, y+j) represents the pixel value in the (2k+1)×(2k+1) window centered on the pixel at (x, y) in the grayscale two-dimensional image, and median represents the median of all pixel values ​​in the window.

[0130] It is understandable that there will be inevitable noise in the image acquisition process, such as that caused by uneven lighting; in order to remove this noise while retaining the edge information in the image, the median filtering algorithm is used for denoising. The principle of median filtering is to replace the value of each pixel in the image with the median of its neighboring pixels, thereby effectively removing isolated noise points without blurring the edges.

[0131] S23: Adaptive threshold binarization processing, including:

[0132] Calculate the adaptive threshold:

[0133]

[0134] In the formula, T(x, y) represents the adaptive threshold of the pixel at the coordinate (x, y), ω represents the size of the binary window, and C represents a constant used to adjust the degree of separation between the foreground and the background;

[0135] Binarization is performed based on the adaptive threshold:

[0136]

[0137] Where I_b(x, y) represents the pixel value after binarization at the coordinate (x, y), 0 represents the background pixel, and 255 represents the foreground pixel.

[0138] It can be understood that the purpose of binarization is to divide the pixels in the image into foreground and background, including the foreground and profile section; the adaptive threshold binarization method dynamically calculates the threshold according to the local area of ​​each pixel, which is suitable for processing images with uneven lighting.

[0139] S24: Connected domain analysis, including:

[0140] Use the breadth-first search algorithm to mark the connected domains of the two-dimensional image after binarization and identify all connected domains;

[0141] Calculate the number of pixels in each connected domain, and calculate the area of ​​each connected domain based on the number of pixels and pixel values;

[0142] The connected domain with the largest area is retained, and the remaining connected domains are set as the background to obtain the preprocessed two-dimensional image.

[0143] Preferably, the calculation formula of the area of ​​the connected domain is specifically:

[0144]

[0145] Among them, A i Represents the area of ​​the i-th connected domain, Region i Represents the coordinate set of all foreground pixels in the i-th connected domain;

[0146] It is understandable that after the adaptive threshold binarization processing, the image will contain multiple connected areas, among which the largest connected area is usually the profile section, and other small connected domains may be noise or background objects. In order to retain only the profile section area, connected domain analysis is required.

[0147] Performing denoising on the three-dimensional point cloud data to obtain pre-processed three-dimensional point cloud data includes:

[0148] For each point in the three-dimensional point cloud data, searching for a set of adjacent points within a preset radius;

[0149] Calculate the mean distance between the point and its adjacent point set. The calculation formula is:

[0150]

[0151] Where, d mean (P i ) represents the mean distance between the i-th point and its adjacent points, N i represents the adjacent point set of the i-th point, P j represents the jth point in the adjacent point set, ||*|| represents the Euclidean distance;

[0152] Traversing each point in the three-dimensional point cloud data to obtain the mean distance between each point and its adjacent point set;

[0153] Calculate the mean and standard deviation of the global point cloud according to the mean distance between each point and its adjacent point set;

[0154] The threshold is calculated according to the mean and standard deviation of the global point cloud, and the calculation formula is:

[0155] τ=μ+α·σ

[0156] In the formula, τ represents the threshold, μ represents the mean of the global point cloud, σ represents the standard deviation of the global point cloud, and σ represents the proportional coefficient;

[0157] Compare the mean distance between each point and its adjacent point set and the threshold in turn. If the mean distance between the point and its adjacent point set is greater than the threshold, the point is removed as an outlier; otherwise, the point is retained.

[0158] S3: inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image;

[0159] Obtaining the trained burr detection model includes:

[0160] Obtaining 3D point cloud data and corresponding 2D images of several profile sections;

[0161] Normalizing the three-dimensional point cloud data and the corresponding two-dimensional images of all the profile sections to obtain the normalized three-dimensional point cloud data and the corresponding two-dimensional images of the profile sections;

[0162] Labeling each point in the normalized three-dimensional point cloud data of all the profile sections into a category to obtain three-dimensional point cloud data of the profile sections with category labels;

[0163] Construct a burr detection model, input the three-dimensional point cloud data with category labels and the corresponding two-dimensional image of the profile section, set the loss function for optimization training, and when the loss function value reaches the minimum, save the corresponding model parameters to obtain the trained burr detection model.

[0164] like Figure 2 As shown, the burr detection model includes a two-dimensional image encoder, a three-dimensional point cloud encoder and a three-dimensional point cloud decoder; wherein the two-dimensional image encoder includes a two-dimensional feature extraction unit; the three-dimensional point cloud encoder includes a first multi-layer perceptron, a first three-dimensional feature downsampling unit, a first feature fusion unit, a second three-dimensional feature downsampling unit, a second feature fusion unit, a third three-dimensional feature downsampling unit, a third feature fusion unit, a fourth three-dimensional feature downsampling unit and a fourth feature fusion unit; the three-dimensional point cloud decoder includes a second multi-layer perceptron, a first three-dimensional feature upsampling unit, a second three-dimensional feature upsampling unit, a third three-dimensional feature upsampling unit, a fourth three-dimensional feature upsampling unit and a third multi-layer perceptron;

[0165] The output end of the two-dimensional feature extraction unit is connected to the first input end of the first feature fusion unit, the second feature fusion unit, the third feature fusion unit, and the fourth feature fusion unit respectively;

[0166] The output end of the first multi-layer perceptron is connected to the input end of the first three-dimensional feature downsampling unit, the output end of the first three-dimensional feature downsampling unit is connected to the second input end of the first feature fusion unit, the output end of the first feature fusion unit is connected to the input end of the second three-dimensional feature downsampling unit, the output end of the second three-dimensional feature downsampling unit is connected to the second input end of the second feature fusion unit, the output end of the second feature fusion unit is connected to the input end of the third three-dimensional feature downsampling unit, the output end of the third three-dimensional feature downsampling unit is connected to the second input end of the third feature fusion unit, the output end of the third feature fusion unit is connected to the input end of the fourth three-dimensional feature downsampling unit, the output end of the fourth three-dimensional feature downsampling unit is connected to the second input end of the fourth feature fusion unit, and the output end of the fourth feature fusion unit is connected to the input end of the second multi-layer perceptron;

[0167] The output end of the second multilayer perceptron is connected to the first input end of the first three-dimensional feature upsampling unit, the output end of the first three-dimensional feature upsampling unit is connected to the first input end of the second three-dimensional feature upsampling unit, the output end of the second three-dimensional feature upsampling unit is connected to the first input end of the third three-dimensional feature upsampling unit, the output end of the third three-dimensional feature upsampling unit is connected to the first input end of the fourth three-dimensional feature upsampling unit, and the output end of the fourth three-dimensional feature upsampling unit is connected to the input end of the third multilayer perceptron;

[0168] The output end of the first feature fusion unit is also connected to the second input end of the fourth three-dimensional feature upsampling unit, the output end of the second feature fusion unit is also connected to the second input end of the third three-dimensional feature upsampling unit, the output end of the third feature fusion unit is also connected to the second input end of the second three-dimensional feature upsampling unit, and the output end of the fourth feature fusion unit is also connected to the second input end of the first three-dimensional feature upsampling unit.

[0169] like Figure 3 As shown, the structures of the first three-dimensional feature downsampling unit, the second three-dimensional feature downsampling unit, the third three-dimensional feature downsampling unit and the fourth three-dimensional feature downsampling unit are the same, and all include a farthest point sampling layer, a K nearest neighbor grouping layer, a neighborhood maximum pooling layer, a first fully connected layer, a first attention layer and a second fully connected layer connected in sequence; the output end of the neighborhood maximum pooling layer is also connected to the output end of the second fully connected layer.

[0170] like Figure 4 As shown, the first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit have the same structure, and all include a first cross attention layer, a first self-attention layer, a second cross attention layer, a second self-attention layer and a third cross attention layer connected in sequence;

[0171] The output end of the two-dimensional feature extraction unit is connected to the first input end of the first cross-attention layer and the second cross-attention layer respectively, and the output end of the second fully connected layer is connected to the second input end of the first cross-attention layer; the output end of the first self-attention layer is also connected to the first input end of the third cross-attention layer.

[0172] like Figure 5 As shown, the first three-dimensional feature upsampling unit, the second three-dimensional feature upsampling unit, the third three-dimensional feature upsampling unit and the fourth three-dimensional feature upsampling unit have the same structure, and all include a third fully connected layer, a fourth fully connected layer, a trilinear interpolation layer, an element-by-element addition layer, a fifth fully connected layer, a third self-attention layer and a sixth fully connected layer;

[0173] The fourth fully connected layer, the trilinear interpolation layer, the element-by-element addition layer, the fifth fully connected layer, the third self-attention layer and the sixth fully connected layer are connected in sequence;

[0174] The input end of the third fully connected layer is connected to the output end of the third cross attention layer, the output end of the third fully connected layer is connected to the second input end of the element-by-element addition layer, and the output end of the element-by-element addition layer is also connected to the output end of the sixth fully connected layer.

[0175] It can be understood that the two-dimensional feature extraction unit is used to extract a two-dimensional image from an input two-dimensional image; in this embodiment, the two-dimensional feature extraction unit is based on the ResNet50 network structure, including an input layer and a convolution layer connected in sequence; the first perceptron is used to extract high-dimensional features from the input three-dimensional point cloud data; the three-dimensional feature downsampling unit downsamples the extracted high-dimensional features in order to reduce the computational complexity and highlight important features; the three-dimensional feature downsampling unit is based on the PointTransformer network structure, which can process irregular three-dimensional point cloud data and capture the long-distance dependency between points through the self-attention mechanism. The feature fusion unit is used to fuse the two-dimensional image features and the high-dimensional features of the three-dimensional point cloud, capture the correlation between the two-dimensional and three-dimensional features through the self-attention mechanism, and further enhance the interaction between the two modal features through the cross-attention mechanism, so that the fused multi-modal feature information is more comprehensive. The three-dimensional feature upsampling unit is used to restore the original three-dimensional point cloud data format. The three-dimensional feature upsampling unit is based on the Point Transformer network structure, and combines the multi-scale jump connection feature information from the three-dimensional feature downsampling unit during the upsampling process to ensure the integrity of multi-level information. Finally, the third multi-layer perceptron outputs the recognition result of each point, that is, whether it is a burr or a normal section.

[0176] It should be noted that, during the training of the burr detection model, the cross entropy loss function is used to measure the difference between the predicted category and the true category, so as to improve the detection accuracy of the burr detection model. The three-dimensional point cloud data with category labels and the corresponding two-dimensional image are input into the burr detection model, the prediction results are calculated through forward propagation, and the model parameters are optimized through back propagation to minimize the loss function; during the training process of this embodiment, the Adam optimization algorithm is adopted, and the learning rate uses the double cosine annealing strategy to ensure the convergence of the model. The double cosine annealing strategy dynamically adjusts the learning rate at different stages of the training process by periodically reducing the learning rate. The specific learning rate change formula is:

[0177]

[0178] Among them, η t is the current learning rate, η min Learning rate, η max Maximum learning rate, T cur is the current training round, T max is the maximum number of training rounds. In this embodiment, the initial learning rate η max Set to 0.001, the minimum learning rate η min Set to 10 -6 .

[0179] It should be noted that, in this embodiment, the number of input channels of the two-dimensional feature extraction unit is 1; the number of input channels of the first multilayer perceptron is 3, and the number of output channels is 32; the number of input channels of the second multilayer perceptron is 512, and the number of output channels is 512; the number of input channels of the third multilayer perceptron is 32, and the number of output channels is 2; the number of neighbor points to be considered for each point in all three-dimensional feature downsampling units is 16, among which, the input dimension of the first three-dimensional feature downsampling unit is 32, and the output size is 1024×64; the input dimension of the second three-dimensional feature downsampling unit is 64, and the output size is 256×128; the input dimension of the third three-dimensional feature downsampling unit is 128, and the output size is 64×256; the input dimension of the fourth three-dimensional feature downsampling unit is 256, and the output size is 16×512. The number of attention heads in the attention layer of all feature fusion units is 8, among which the feature dimension of each attention head in the first feature fusion unit is 64, the feature dimension of each attention head in the second feature fusion unit is 128, the feature dimension of each attention head in the third feature fusion unit is 256, and the feature dimension of each attention head in the fourth feature fusion unit is 512. The output size of the first three-dimensional feature upsampling unit is 64×256, the output scale of the second three-dimensional feature upsampling unit is 256×128, the output scale of the third three-dimensional feature upsampling unit is 1024×64, and the output scale of the fourth three-dimensional feature upsampling unit is 4096×32.

[0180] It should be noted that after the preprocessed 2D image and 3D point cloud data are input into the trained burr detection model, the burr label of each point in the preprocessed 3D point cloud data is first identified, that is, there is a burr or a normal section; then the 3D point cloud data is projected onto the imaging plane of the 2D image in the 3D space, specifically:

[0181] According to the posture information of the two-dimensional industrial camera, the points of the three-dimensional point cloud data are associated with the pixels of the two-dimensional image through the camera projection equation to achieve preliminary pairing. The camera projection equation is:

[0182] p=K 2D [R 2D |t 2D ]P

[0183] Among them, p represents the pixel coordinates in the two-dimensional image, K 2D is the internal parameter matrix of the two-dimensional industrial camera, including focal length and principal point offset, R 2D and t 2D are the external parameters of the industrial camera, representing the two-dimensional camera rotation matrix and the two-dimensional camera translation vector respectively, and P is the homogeneous coordinate of the point in the three-dimensional point cloud data;

[0184] Afterwards, for each projection point of the 3D point cloud data projected onto the imaging plane, the Euclidean distance to each pixel point is calculated, and the pixel point with the smallest Euclidean distance is found as the closest pixel point;

[0185] Finally, the burr label of the point cloud point corresponding to the projection point is transferred to the nearest pixel point as the burr label of the pixel point. Finally, each pixel point on the two-dimensional graph has a burr label with burrs or normal sections, and the burr area in the two-dimensional image is obtained.

[0186] S4: removing the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured;

[0187] S5: Correcting the burr-free image of the cross section of the profile to be measured to obtain a corrected image of the cross section of the profile to be measured, and measuring dimensional parameters on the corrected image of the cross section of the profile to be measured.

[0188] It should be noted that due to the physical characteristics of camera imaging and environmental factors in the measurement scene, such as the distance and angle between the camera and the target object, certain geometric distortion will be introduced into the image. In order to ensure the measurement accuracy, a single hard matrix is ​​used to correct the collective error. The homography matrix transforms a plane projection in the burr-free image into another plane to obtain the corrected image of the profile section to be measured. The transformation relationship is:

[0189]

[0190] Where H is a 3×3 homography matrix, (x, y) and (x ′ ,y ′ ) represent the pixel coordinates before and after correction respectively;

[0191] Then, edge detection is performed on the corrected image of the profile section to be measured using an edge detection algorithm to obtain a set of contour lines of the profile section to be measured;

[0192] Arbitrarily select a preset measurement position, and select two corresponding target contour lines from the contour line set of the profile section to be measured;

[0193] Calculating the pixel distance between two target contour lines in the corrected image of the cross section of the profile to be measured;

[0194] Calculate the actual distance between the two target contour lines according to the preset proportionality coefficient and the pixel distance, and use the actual distance as the actual size of the preset measurement part;

[0195] The preset measuring parts are traversed to obtain the actual sizes of all the preset measuring parts, and the size parameters of the calibrated image of the cross section of the profile to be measured are formed.

[0196] It can be understood that the edge results of the edge detection algorithm are a series of boundary lines or curves. These edge results are linearly fitted through Hough transform to obtain the contour line set of each key part of the profile section to be measured; and the pixel distance is usually completed by calculating the vertical distance between the two target contour lines at a certain measurement point. The calculation formula is:

[0197]

[0198] Among them, A1, B1, C1 and A2, B2, C2 are the parameters of the two target contour lines, and (x, y) are the coordinates of the measurement points;

[0199] Finally, the pixel distance is converted into the actual size according to the proportional relationship between the pixel distance and the three-dimensional space. The proportional relationship is the proportional coefficient obtained according to the calibration experiment, which represents the actual physical length represented by each pixel. The calculation formula of the actual size is:

[0200] d real =α×d pixels

[0201] In the formula, α represents the proportionality coefficient.

[0202] In actual use, during the inspection process of the production line, it can quickly complete the automatic measurement of the profile section size and significantly reduce the measurement error caused by manual operation. In particular, for the aluminum profile section with burrs, the burr area can be accurately detected and removed to ensure the accuracy of the size measurement.

[0203] Example 3

[0204] This embodiment provides a profile cross-sectional dimension measurement system based on two-dimensional images and three-dimensional point clouds, which is used in the method described in Embodiment 1 or 2, such as Figure 6 As shown, including:

[0205] A data acquisition module, used to acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image;

[0206] A data processing module, used to pre-process the two-dimensional image and the three-dimensional point cloud data respectively, and obtain the pre-processed two-dimensional image and three-dimensional point cloud data accordingly;

[0207] A burr detection module, used for inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image;

[0208] A burr removal module, used to remove the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured;

[0209] The dimension measurement module is used to correct the burr-free image of the cross section of the profile to be measured, obtain the corrected image of the cross section of the profile to be measured, and measure the dimension parameters on the corrected image of the cross section of the profile to be measured.

[0210] The same or similar reference numerals correspond to the same or similar components;

[0211] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0212] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds, characterized in that: include: Acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image; Preprocessing the two-dimensional image and the three-dimensional point cloud data respectively to obtain the preprocessed two-dimensional image and three-dimensional point cloud data respectively; Inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image; Removing the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured; The burr-free image of the cross section of the profile to be measured is corrected to obtain a corrected image of the cross section of the profile to be measured, and the dimensional parameters are measured on the corrected image of the cross section of the profile to be measured.

2. The profile cross-sectional dimension measurement method based on two-dimensional image and three-dimensional point cloud according to claim 1, characterized in that: The step of obtaining a two-dimensional image and a depth image of the cross section of the profile to be measured comprises: The profile section to be measured is fixed, a two-dimensional image of the profile section to be measured is obtained by using a two-dimensional industrial camera, a depth image of the profile section to be measured is obtained by using a three-dimensional depth camera, and the position and posture information of the three-dimensional depth camera and the two-dimensional industrial camera are saved.

3. The profile cross-sectional dimension measurement method based on two-dimensional image and three-dimensional point cloud according to claim 1, characterized in that: Generating three-dimensional point cloud data according to the depth image includes: The formula for generating 3D point cloud data from a depth image is: Where, X i ,Y i ,Z i Represents the 3D coordinates of the i-th point cloud in the 3D point cloud data, u i ,v i Represents the two-dimensional coordinates of the i-th pixel in the depth image, depth(u i ,v i ) represents the depth value of the i-th pixel, c x ,c y represents the optical center coordinates of the 3D depth camera, f x ,f y Represent the focal length of the 3D depth camera in the x and y directions respectively.

4. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 1, characterized in that: Obtaining the trained burr detection model includes: Obtaining 3D point cloud data and corresponding 2D images of several profile sections; Normalizing the three-dimensional point cloud data and the corresponding two-dimensional images of all the profile sections to obtain the normalized three-dimensional point cloud data and the corresponding two-dimensional images of the profile sections; Labeling each point in the normalized three-dimensional point cloud data of all the profile sections into a category to obtain three-dimensional point cloud data of the profile sections with category labels; Construct a burr detection model, input the three-dimensional point cloud data with category labels and the corresponding two-dimensional image of the profile section, set the loss function for optimization training, and when the loss function value reaches the minimum, save the corresponding model parameters to obtain the trained burr detection model.

5. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 4, characterized in that: The burr detection model includes a two-dimensional feature extraction unit, a first multi-layer perceptron, a first three-dimensional feature down-sampling unit, a first feature fusion unit, a second three-dimensional feature down-sampling unit, a second feature fusion unit, a third three-dimensional feature down-sampling unit, a third feature fusion unit, a fourth three-dimensional feature down-sampling unit, a fourth feature fusion unit, a second multi-layer perceptron, a first three-dimensional feature up-sampling unit, a second three-dimensional feature up-sampling unit, a third three-dimensional feature up-sampling unit, a fourth three-dimensional feature up-sampling unit and a third multi-layer perceptron; The output end of the two-dimensional feature extraction unit is connected to the first input end of the first feature fusion unit, the second feature fusion unit, the third feature fusion unit, and the fourth feature fusion unit respectively; The output end of the first multi-layer perceptron is connected to the input end of the first three-dimensional feature downsampling unit, the output end of the first three-dimensional feature downsampling unit is connected to the second input end of the first feature fusion unit, the output end of the first feature fusion unit is connected to the input end of the second three-dimensional feature downsampling unit, the output end of the second three-dimensional feature downsampling unit is connected to the second input end of the second feature fusion unit, the output end of the second feature fusion unit is connected to the input end of the third three-dimensional feature downsampling unit, the output end of the third three-dimensional feature downsampling unit is connected to the second input end of the third feature fusion unit, the output end of the third feature fusion unit is connected to the input end of the fourth three-dimensional feature downsampling unit, the output end of the fourth three-dimensional feature downsampling unit is connected to the second input end of the fourth feature fusion unit, and the output end of the fourth feature fusion unit is connected to the input end of the second multi-layer perceptron; The output end of the second multilayer perceptron is connected to the first input end of the first three-dimensional feature upsampling unit, the output end of the first three-dimensional feature upsampling unit is connected to the first input end of the second three-dimensional feature upsampling unit, the output end of the second three-dimensional feature upsampling unit is connected to the first input end of the third three-dimensional feature upsampling unit, the output end of the third three-dimensional feature upsampling unit is connected to the first input end of the fourth three-dimensional feature upsampling unit, and the output end of the fourth three-dimensional feature upsampling unit is connected to the input end of the third multilayer perceptron; The output end of the first feature fusion unit is also connected to the second input end of the fourth three-dimensional feature upsampling unit, the output end of the second feature fusion unit is also connected to the second input end of the third three-dimensional feature upsampling unit, the output end of the third feature fusion unit is also connected to the second input end of the second three-dimensional feature upsampling unit, and the output end of the fourth feature fusion unit is also connected to the second input end of the first three-dimensional feature upsampling unit.

6. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 5, characterized in that: The structures of the first three-dimensional feature downsampling unit, the second three-dimensional feature downsampling unit, the third three-dimensional feature downsampling unit and the fourth three-dimensional feature downsampling unit are similar, and all include a farthest point sampling layer, a K nearest neighbor grouping layer, a neighborhood maximum pooling layer, a first fully connected layer, a first attention layer and a second fully connected layer connected in sequence; the output end of the neighborhood maximum pooling layer is also connected to the output end of the second fully connected layer.

7. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 6, characterized in that: The first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit have the same structure, and all include a first cross attention layer, a first self-attention layer, a second cross attention layer, a second self-attention layer and a third cross attention layer connected in sequence; The output end of the two-dimensional feature extraction unit is connected to the first input end of the first cross-attention layer and the second cross-attention layer respectively, and the output end of the second fully connected layer is connected to the second input end of the first cross-attention layer; the output end of the first self-attention layer is also connected to the first input end of the third cross-attention layer.

8. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 7, characterized in that: The first three-dimensional feature upsampling unit, the second three-dimensional feature upsampling unit, the third three-dimensional feature upsampling unit and the fourth three-dimensional feature upsampling unit have the same structure, and all include a third fully connected layer, a fourth fully connected layer, a trilinear interpolation layer, an element-by-element addition layer, a fifth fully connected layer, a third self-attention layer and a sixth fully connected layer; The fourth fully connected layer, the trilinear interpolation layer, the element-by-element addition layer, the fifth fully connected layer, the third self-attention layer and the sixth fully connected layer are connected in sequence; The input end of the third fully connected layer is connected to the output end of the third cross attention layer, the output end of the third fully connected layer is connected to the second input end of the element-by-element addition layer, and the output end of the element-by-element addition layer is also connected to the output end of the sixth fully connected layer.

9. The method for measuring profile cross-sectional dimensions based on two-dimensional images and three-dimensional point clouds according to claim 1, characterized in that: Measuring the dimensional parameters on the calibrated image of the cross section of the profile to be measured includes: Performing edge detection on the corrected image of the profile section to be measured using an edge detection algorithm to obtain a set of contour lines of the profile section to be measured; Arbitrarily select a preset measurement position, and select two corresponding target contour lines from the contour line set of the profile section to be measured; Calculating the pixel distance between two target contour lines in the corrected image of the cross section of the profile to be measured; Calculate the actual distance between the two target contour lines according to the preset proportionality coefficient and the pixel distance, and use the actual distance as the actual size of the preset measurement part; The preset measuring parts are traversed to obtain the actual sizes of all the preset measuring parts, and the size parameters of the calibrated image of the cross section of the profile to be measured are formed.

10. A profile cross-sectional dimension measurement system based on two-dimensional images and three-dimensional point clouds, used to implement the method according to any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to acquire a two-dimensional image and a depth image of the cross section of the profile to be measured, and generate three-dimensional point cloud data according to the depth image; A data processing module, used to preprocess the two-dimensional image and the three-dimensional point cloud data respectively, and obtain the preprocessed two-dimensional image and three-dimensional point cloud data accordingly; A burr detection module, used for inputting the preprocessed two-dimensional image and three-dimensional point cloud data into a trained burr detection model to obtain a burr area in the two-dimensional image; A burr removal module, used to remove the burr area on the two-dimensional image to obtain a burr-free image of the cross section of the profile to be measured; The dimension measurement module is used to correct the burr-free image of the cross section of the profile to be measured, obtain the corrected image of the cross section of the profile to be measured, and measure the dimension parameters on the corrected image of the cross section of the profile to be measured.

Images