Three-dimensional point cloud virtual assembling method based on image recognition

Abstract

The invention belongs to the technical field of building construction, and particularly relates to a three-dimensional point cloud virtual assembling method based on image recognition. A three-dimensional point cloud virtual splicing method based on image recognition comprises the steps that point cloud data of splicing plates and side cross beam splicing plates at the joints of side main beams, side box beams and steel truss web plates are collected through a base station type scanner, standard size frame lines are constructed by conducting manual segmentation, voxel filtering and Gaussian filtering on the point cloud data, an image is intercepted, and the three-dimensional point cloud virtual splicing method based on image recognition is obtained. Gray processing, threshold segmentation, perspective transformation and size identification are carried out on the image, two-dimensional coordinates of bolt hole groups and frame lines are obtained, a two-dimensional to three-dimensional coordinate system is constructed, three-dimensional coordinates in the bolt hole groups are obtained through conversion, and virtual splicing is carried out according to the corresponding relation between the bolt hole groups of different components. According to the method, the efficiency and the precision of virtual assembly are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of building construction, and particularly relates to a three-dimensional point cloud virtual assembly method based on image recognition. Background Art

[0002] During the process of assembling steel truss girder segments of large steel truss girder bridges in a scattered manner, pre-assembly is usually required to ensure the assembly matching performance of the girder segments and improve the installation quality of the girder segments. The pre-assembly of the girder segments is usually carried out in the steel girder yard in a 1+1 or N+1 assembly mode. The assembly requires repeated assembly and disassembly, consuming a large amount of human, equipment resources and time costs. The main girders of extra-large suspension bridges in mountainous areas usually adopt a steel truss girder structure and need to be hoisted after being assembled on-site in a scattered manner. Construction workers need to strictly follow the assembly sequence according to the design drawings and construction plans. During the assembly process, first, the main truss members need to be precisely aligned through splicing plates, and then temporary drift pins are used for fixation to ensure the preliminary connection between the members. Next, the construction team will gradually increase the number of drift pins to enhance the connection stability. Subsequently, these drift pins will be gradually replaced with pre-designed high-strength bolts. Finally, after the main splicing of the truss girder is completed, the chord members and the main truss also need to be welded to ensure the strength and durability of the entire bridge.

[0003] Due to the complex mountain environment and narrow site, the scattered assembly process usually can only carry out 1+1 trial assembly, and the erection speed of the steel truss girder is greater than the trial assembly speed. Therefore, the trial assembly has become a key process affecting the construction of the steel girder. However, in the actual construction process, the assembly of steel truss girder bridges faces many challenges. Due to the possible dimensional errors during the steel processing and transportation, some bolt holes may not be completely aligned with the holes on the splicing plates, resulting in the need for reaming or adjustment by construction workers during the assembly. This not only increases the work difficulty but also may affect the stress distribution of the plate members. Improper stress distribution will increase the fatigue of the structure and reduce the safety and service life of the bridge.

[0004] In recent years, with the rapid development of model technology and precise measurement technology, the assembly data of precast components can be processed through image digital processing, precise measurement of components, reverse modeling, and computer simulation analysis. The idea of replacing trial assembly with computer virtual assembly has been proposed. The virtual assembly of truss segments aims to obtain the appearance dimension model of the structure and use virtual simulation technology for assembly simulation. Therefore, after the scattered assembly of the members is completed, a reverse model of the truss segment needs to be obtained. In view of the characteristics of the large scale of the steel truss beam truss segment of the suspension bridge and the high precision requirement for the measurement of bolt hole dimensions, a base station scanner is used to perform an overall scan of the truss segment. This scanner is similar to a total station and can quickly obtain the overall point cloud model of the truss segment by setting multiple scanning stations. The accuracy of the appearance contour model of the truss segment reaches the millimeter level, but the hole group information cannot be accurately identified. Using image recognition to obtain bolt hole group information such as coordinates and hole diameters is a mainstream image information extraction method. However, there are the following problems in obtaining bolt hole coordinates in this way: it is necessary to paste a target and a frame line on the component as a dimension reference, the operation is cumbersome and it is impossible to paste on the high places of large components to collect information; it is easily affected by the photo quality and it is impossible to collect high-precision pictures at any time, and the photo quality is affected by conditions such as lighting. Summary of the Invention

[0005] Aiming at the technical problems of the 1+1 pre-assembly of the truss segment of the steel truss beam bridge in mountainous areas in the prior art, which has high requirements for the site, large consumption of process resources, complex preliminary preparation work required for image recognition, and is easily affected by on-site conditions, while point cloud recognition cannot construct the bolt hole feature lattice, the object of the present invention is to provide a three-dimensional point cloud virtual assembly method based on image recognition. This method manually segments, voxel filters, and Gaussian filters the point cloud data to construct a standard size frame line and intercepts images, performs gray processing, threshold segmentation, perspective transformation, and size recognition on the images to obtain the two-dimensional coordinates of the bolt hole group and the frame line, constructs a two-dimensional to three-dimensional coordinate system, converts to obtain the internal three-dimensional coordinates of the bolt hole group, and performs virtual assembly according to the corresponding relationship between the bolt hole groups of different components, significantly improving the efficiency and accuracy of virtual assembly.

[0006] The technical solution of the present invention is as follows: A three-dimensional point cloud virtual assembly method based on image recognition, comprising the following steps: S1: Collect point cloud data of the splicing plates at the joints of the side main beams, side box girders, steel truss beam webs, and side crossbeam splicing plates using a base station scanner; S2: For the point cloud data collected in step S1, perform trimming and segmentation, extract the point cloud data containing the bolt hole group as a control point, and perform voxel filtering and Gaussian filtering processing; S3: Use the random sample consensus algorithm to fit the plane of the component / splicing plate for the point cloud data processed in step S2, and intercept a two-dimensional image, and perform image preprocessing on the two-dimensional image, specifically including gray processing, threshold segmentation, and perspective transformation; S4: For the front view of the bolt holes in the two-dimensional image after perspective transformation, use the Hough transform to detect circles. By converting pixels to actual dimensions, set the minimum diameter and maximum diameter thresholds to identify the diameter and center coordinates of the bolt hole group. Based on the three-dimensional coordinate information and two-dimensional coordinate information of the frame line, construct a coordinate transformation matrix from two dimensions to three dimensions to generate the three-dimensional coordinates of the bolt hole group; S5: Repeat steps S1 to S4 to obtain the bolt hole control point groups of the splicing plate and the parts to be spliced of the component respectively. Generate a coordinate transformation matrix according to the three-dimensional coordinates of the corresponding bolt holes of the component to be spliced, and apply the transformation matrix to transform the complete component for virtual trial splicing.

[0007] In step S1, the open3d.visualization.draw_geometries_with_editing function is used to clip and segment the point cloud data, and the open3d.visualization.draw_geometries_with_editing function is a function in open3d.

[0008] For the point cloud data of the bolt hole group in step S2, voxel filtering and Gaussian filtering are performed. The specific process is as follows: voxel filtering is performed through the open3d.voxel_down_sample function. In voxel filtering, adjust the sampling radius and voxel size to set the range of adjacent points and the point cloud density. Gaussian filtering is performed through filter_gaussian(search_param=open3d.geometry.KDTreeSearchParamHybrid(radius=0.01,max_nn=30),stddev=2.0). In Gaussian filtering, adjust the standard deviation parameter to control the width of the Gaussian function and maintain the balance between the smoothing effect and the retention of details. The open3d.voxel_down_sample function and the filter_gaussian(search_param=open3d.geometry.KDTreeSearchParamHybrid(radius=0.01,max_nn=30),stddev=2.0) function are functions in open3d.

[0009] In step S3, the random sample consensus algorithm is used to fit the plane of the component / splicing plate for the point cloud data processed in step S2. A square frame line with a set standard size is generated on the plane, and a red disk is randomly generated within the square frame line to intercept the two-dimensional image.

[0010] In step S3, the random sample consensus algorithm is used to fit the plane of the component / splice plate. The specific process is as follows: The cloud.segment_plane function is used for RANSAC plane segmentation. The maximum distance from a point to the fitted plane and the distance threshold of the point are distance_threshold = 0.01. The number of randomly selected points in each iteration is ransac_n = 3, and the number of iterations is num_iterations = 500. The plane model parameters of the fitted plane are represented as ax + by + cz + d = 0, where [a, b, c] is the normal vector of the plane and d is the distance from the plane to the origin.

[0011] In step S3, the grayscale processing is as follows: The cv2.cvtColor function is used to perform grayscale processing on the image to obtain a grayscale image. Then, the cv2.GaussianBlur function is used to perform Gaussian blur processing on the grayscale image, and the cv2.threshold function is used to perform binary processing. The cv2.cvtColor function, cv2.GaussianBlur function, and cv2.threshold function are functions in the OpenCV library.

[0012] In step S3, the perspective transformation is as follows: The pers_img_squar(img, src_list, length) function is used to identify the coordinates of the four corner points of the frame line in the two-dimensional image as the source points pts1 for perspective transformation. The target points pts2 are defined according to the side length length of the target image. The perspective transformation matrix is calculated based on the source points and the target points. According to the source image and the perspective transformation matrix, the source image img is subjected to perspective transformation to obtain the front view of the perspective-transformed image. The pers_img_squar(img, src_list, length) function is a function in the openCV library.

[0013] In step S4, circles are detected by the Hough transform cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1 = 40, param2 = 20, minRadius = 20, maxRadius = 28), and two-dimensional center coordinates are output. The least squares method np.linalg.lstsq(A, B, rcond = None)[0] is used to solve the transformation matrix from two dimensions to three dimensions, and the two-dimensional coordinates of the bolt holes are transformed into three-dimensional coordinates. The cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1 = 40, param2 = 20, minRadius = 20, maxRadius = 28) function is a function in the OpenCV library.

[0014] In step S5, the compute_transformation_matrix(source_points, target_points) function is constructed. The point cloud is converted into a skspatial Points object. The rotation matrix is calculated through U, S, Vt = np.linalg.svd(covariance_matrix), R = Vt.T @ U.T. The translation variable is calculated through t = centroid_target - R @centroid_source. The translation and rotation matrices are combined into a homogeneous transformation matrix through transformation_matrix = np.eye(4), transformation_matrix[:3, :3] = R, transformation_matrix[:3, 3]= t. This matrix is called to perform coordinate transformation on the two point clouds to obtain the assembled finished product.

[0015] The OpenCV library described in the present invention is a cross-platform computer vision and machine learning software library distributed based on the Apache2.0 license (open source). It can run on operating systems such as Linux, Windows, Android, and Mac OS. It is lightweight and efficient - composed of a series of C functions and a small number of C++ classes, and at the same time provides interfaces in languages such as Python, Ruby, and MATLAB, implementing many general algorithms in image processing and computer vision.

[0016] The technical effect of the present invention is as follows: manually segment the point cloud data, perform voxel filtering and Gaussian filtering to construct a standard-size frame line, intercept the image, perform gray processing, threshold segmentation, perspective transformation, and size recognition on the image to obtain the two-dimensional coordinates of the bolt hole group and the frame line, construct a two-dimensional to three-dimensional coordinate system, convert to obtain the internal three-dimensional coordinates of the bolt hole group, and perform virtual assembly according to the corresponding relationship between the bolt hole groups of different components, significantly improving the efficiency and accuracy of virtual assembly.

[0017] The following will be further described with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a three-dimensional point cloud virtual assembly method based on image recognition according to an embodiment of the present invention.

[0019] Figure 2 is a point cloud data diagram of the crossbeam HL2AASZ-25 according to an embodiment of the present invention.

[0020] Figure 3 is a point cloud data diagram of the side box SB10A2aZ according to an embodiment of the present invention.

[0021] Figure 4 is a control point area diagram of the bolt hole for generating a reference frame line according to an embodiment of the present invention.

[0022] Figure 5 is a perspective transformation and output diagram of the center coordinates and circle size according to an embodiment of the present invention.

[0023] Figure 6 is a comparison diagram of the bolt holes (black circles) obtained by identification and conversion and the scanned bolt holes according to an embodiment of the present invention.

[0024] Figure 7 is a virtual assembly diagram of the side box - splicing plate according to an embodiment of the present invention.

[0025] Figure 8 is a virtual assembly diagram of the side box - splicing plate - side box - splicing plate - crossbeam according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS Embodiment 1

[0026] As Figure 1 shown, a three-dimensional point cloud virtual assembly method based on image recognition includes the following steps: S1: Collect point cloud data of the splicing plates at the joints of the side main girder, side box girder, web of the steel truss girder, and side crossbeam splicing plate using a base station scanner; S2: For the point cloud data collected in step S1, perform trimming and segmentation, extract the point cloud data containing the bolt hole group as the control point, and perform voxel filtering and Gaussian filtering processing; S3: Apply the random sample consensus algorithm to fit the plane of the component / splicing plate to the point cloud data processed in step S2, intercept a two-dimensional image, and perform image preprocessing on the two-dimensional image, specifically including grayscale processing, threshold segmentation, and perspective transformation; S4: For the front view of the bolt holes in the two-dimensional image after perspective transformation, use the Hough transform to detect circles. By converting the pixels and the actual dimensions, set the minimum diameter and maximum diameter thresholds to identify the diameter and center coordinates of the bolt hole group. Based on the three-dimensional coordinate information and two-dimensional coordinate information of the frame line, construct a coordinate transformation matrix from two dimensions to three dimensions to generate the three-dimensional coordinates of the bolt hole group; S5: Repeat steps S1 - S4 to obtain the bolt hole control point groups of the splicing plate and the component to be assembled respectively. Generate a coordinate transformation matrix according to the three-dimensional coordinates of the corresponding bolt holes of the component to be assembled, and apply the transformation matrix to transform the complete component for virtual trial assembly.

[0027] In step S1, the open3d.visualization.draw_geometries_with_editing function is used to clip and segment the point cloud data, and the open3d.visualization.draw_geometries_with_editing function is a function in open3d.

[0028] For the point cloud data of the bolt hole group in step S2, perform voxel filtering and Gaussian filtering. The specific process is as follows: perform voxel filtering through the open3d.voxel_down_sample function. In voxel filtering, adjust the sampling radius and voxel size to set the range of neighboring points and the point cloud density. Perform Gaussian filtering through filter_gaussian(search_param = open3d.geometry.KDTreeSearchParamHybrid(radius = 0.01, max_nn = 30), stddev = 2.0). In Gaussian filtering, adjust the standard deviation parameter to control the width of the Gaussian function and maintain the balance between the smoothing effect and the retention of details. The open3d.voxel_down_sample function and the filter_gaussian(search_param = open3d.geometry.KDTreeSearchParamHybrid(radius = 0.01, max_nn = 30), stddev = 2.0) function are functions in open3d.

[0029] In step S3, the point cloud data processed in step S2 is fitted to the plane of the component / splice plate using the random sample consensus algorithm. A square frame line of a set standard size is generated on the plane, and a red disc is randomly generated within the square frame line to intercept a two-dimensional image.

[0030] In step S3, the random sample consensus algorithm is used to fit the plane of the component / splice plate. The specific process is as follows: The cloud.segment_plane function is used for RANSAC plane segmentation. The maximum distance from the point to the fitted plane and the distance threshold of the point are distance_threshold = 0.01. The number of randomly selected points in each iteration is ransac_n = 3, and the number of iterations is num_iterations = 500. The plane model parameters of the fitted plane are represented as ax + by + cz + d = 0, where [a, b, c] is the normal vector of the plane and d is the distance from the plane to the origin.

[0031] In step S3, the grayscale processing is as follows: The cv2.cvtColor function is used to perform grayscale processing on the image to obtain a grayscale image. Then, the cv2.GaussianBlur function is used to perform Gaussian blur processing on the grayscale image, and the cv2.threshold function is used to perform binarization processing. The cv2.cvtColor function, cv2.GaussianBlur function, and cv2.threshold function are functions in the OpenCV library.

[0032] In step S3, the perspective transformation is as follows: The coordinates of the four frame line corner points of the two-dimensional image in the image are identified by the pers_img_squar(img, src_list, length) function as the source points pts1 for perspective transformation. The target points pts2 are defined according to the side length length of the target image. The perspective transformation matrix is calculated based on the source points and the target points. According to the source image and the perspective transformation matrix, the source image img is subjected to perspective transformation to obtain the front view of the perspective-transformed image. The pers_img_squar(img, src_list, length) function is a function in the OpenCV library.

[0033] In step S4, circles are detected by the Hough transform cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1 = 40, param2 = 20, minRadius = 20, maxRadius = 28), and the two-dimensional center coordinates are output. The least squares method np.linalg.lstsq(A, B, rcond = None)[0] is used to solve the transformation matrix from two dimensions to three dimensions, and the two-dimensional coordinates of the bolt holes are transformed into three-dimensional coordinates. The cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1 = 40, param2 = 20, minRadius = 20, maxRadius = 28) function is a function in the OpenCV library.

[0034] In step S5, the compute_transformation_matrix(source_points, target_points) function is constructed to convert the point cloud into a Points object of skspatial. The rotation matrix is calculated by U, S, Vt = np.linalg.svd(covariance_matrix), R = Vt.T @ U.T, the translation variable is calculated by t = centroid_target - R @ centroid_source, and the translation and rotation matrices are combined into a homogeneous transformation matrix by transformation_matrix = np.eye(4), transformation_matrix[:3, :3] = R, transformation_matrix[:3, 3] = t. This matrix is called to perform coordinate transformation on the two point clouds to obtain the assembled finished product. Example 2

[0035] A three-dimensional point cloud virtual assembly method based on image recognition as in Example 1 is used to perform pre-construction scanning and virtual assembly operations on the crossbeam HL2AASZ-25 and side box SB10A2aZ, the component parts of the Anluo Yellow River Bridge, in the steel component factory. The specific process is as follows: S1: The point cloud data of the crossbeam HL2AASZ-25 and side box SB10A2aZ are collected by a base station scanner, as Figure 2 and Figure 3 shown; S2: The point cloud data collected in step S1 are trimmed and segmented to extract the point cloud data containing the bolt hole group as the control points, as Figure 4 shown, and voxel filtering and Gaussian filtering are performed. S3: Apply the random sample consensus algorithm to fit the plane of the component / splice plate for the point cloud data processed in step S2, intercept a two-dimensional image, and perform image preprocessing on the two-dimensional image, specifically including grayscale processing, threshold segmentation, and perspective transformation; S4: For the front view of the bolt holes in the two-dimensional image after perspective transformation, use the Hough transform to detect circles. By converting the pixel and actual dimensions, set the minimum diameter and maximum diameter thresholds to identify the diameter and center coordinates of the bolt hole group. Based on the three-dimensional coordinate information and two-dimensional coordinate information of the frame line, construct a coordinate transformation matrix from two dimensions to three dimensions to generate the three-dimensional coordinates of the bolt hole group, as Figure 5 and shown in Table 1; Table 1 Assembly corner point coordinates

[0036] S5: Generate a three-dimensional point cloud ring of a given size on the converted three-dimensional bolt hole coordinates and compare it with the scanned component point cloud. The effect is as Figure 6 shown. It can be seen from Figure 6 that the bolt hole coordinates obtained by the present invention are basically consistent with the measured bolt hole coordinates. The small amount of error mainly comes from the hyperparameters set in the Hough circle detection and the distortion generated during the point cloud preprocessing process. However, during the registration process, multiple bolt hole positions are aligned, so the small amount of error will not affect the assembly effect; Repeat this process to obtain the bolt hole coordinates of the component to be assembled, calculate the rigid body transformation matrix, and call this matrix to perform coordinate transformation on the two pieces of point cloud to obtain the assembled finished product, as Figure 7 、 Figure 8 shown.

[0037] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A three-dimensional point cloud virtual assembly method based on image recognition, characterized in that: It includes the following steps: S1: Use a base station scanner to collect point cloud data of the splicing plates at the joints of the side main girders, side box girders, steel truss girder webs, and side crossbeam splicing plates; S2: For the point cloud data collected in step S1, perform clipping and segmentation, extract the point cloud data containing the bolt hole groups as control points, and perform voxel filtering and Gaussian filtering; S3: Use the random sample consensus algorithm to fit the plane of the component / splicing plate for the point cloud data processed in step S2, intercept a two-dimensional image, and perform image preprocessing on the two-dimensional image, specifically including grayscale processing, threshold segmentation, and perspective transformation; S4: For the front view of the bolt holes in the two-dimensional image after perspective transformation, use the Hough transform to detect circles, set the minimum diameter and maximum diameter thresholds through the conversion between pixels and actual sizes, identify the diameter and center coordinates of the bolt hole group, and based on the three-dimensional coordinate information and two-dimensional coordinate information of the frame line, construct a coordinate transformation matrix from two dimensions to three dimensions to generate the three-dimensional coordinates of the bolt hole group; S5: Repeat steps S1 to S4 to obtain the bolt hole control point groups of the splicing plates and the parts to be spliced of the components respectively, generate a coordinate transformation matrix according to the three-dimensional coordinates of the corresponding bolt holes of the components to be spliced, and apply the transformation matrix to transform the complete components for virtual trial assembly.

2. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, wherein: In step S1, the open3d.visualization.draw_geometries_with_editing function is used to perform clipping and segmentation on the point cloud data, and the open3d.visualization.draw_geometries_with_editing function is a function in open3d.

3. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, wherein: For the point cloud data of the bolt hole groups in step S2, perform voxel filtering and Gaussian filtering. The specific process is as follows: perform voxel filtering through the open3d.voxel_down_sample function. In voxel filtering, adjust the sampling radius and voxel size to set the range of neighboring points and the point cloud density. Perform Gaussian filtering through filter_gaussian(search_param=open3d.geometry.KDTreeSearchParamHybrid(radius=0.01,max_nn=30),stddev=2.0). In Gaussian filtering, adjust the standard deviation parameter to control the width of the Gaussian function and maintain the balance between the smoothing effect and the retention of details. The open3d.voxel_down_sample function and the filter_gaussian(search_param=open3d.geometry.KDTreeSearchParamHybrid(radius=0.01,max_nn=30),stddev=2.0) function are functions in open3d.

4. The 3D point cloud virtual assembly method based on image recognition according to claim 1, characterized in that: In step S3, the point cloud data processed in step S2 is fitted to the plane of the component / splice plate using the Random Sample Consensus (RANSAC) algorithm. A square frame line of a set standard size is generated on the plane, and a red disc is randomly generated within the range of the square frame line to intercept a two-dimensional image.

5. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, characterized in that: In step S3, the plane of the component / splice plate is fitted using the Random Sample Consensus (RANSAC) algorithm. The specific process is as follows: The cloud.segment_plane function is used for RANSAC plane segmentation. The maximum distance from a point to the fitted plane and the distance threshold of the point are distance_threshold = 0.

01. The number of randomly selected points in each iteration is ransac_n = 3, and the number of iterations is num_iterations = 500. The plane model parameters of the fitted plane are represented as ax + by + cz + d = 0, where [a, b, c] is the normal vector of the plane and d is the distance from the plane to the origin. The cloud.segment_plane function is a function in open3d.

6. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, wherein: In step S3, the grayscale processing is as follows: The cv2.cvtColor function is used to perform grayscale processing on the image to obtain a grayscale image. Then, the cv2.GaussianBlur function is used to perform Gaussian blur processing on the grayscale image, and the cv2.threshold function is used to perform binary processing. The cv2.cvtColor function, cv2.GaussianBlur function, and cv2.threshold function are functions in the OpenCV library.

7. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, wherein: In step S3, the perspective transformation is as follows: The coordinates of the four corner points of the frame line of the two-dimensional image are identified in the image by the pers_img_squar(img, src_list, length) function as the source points pts1 for perspective transformation. The target points pts2 are defined according to the side length length of the target image. The perspective transformation matrix is calculated based on the source points and the target points. Based on the source image and the perspective transformation matrix, the source image img is subjected to perspective transformation to obtain the front view of the perspective-transformed image. The pers_img_squar(img, src_list, length) function is a function in the openCV library.

8. The 3D point cloud virtual assembly method based on image recognition according to claim 1, characterized in that: In step S4, circles are detected by the Hough transform cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1=40, param2=20, minRadius=20, maxRadius=28), and the two-dimensional center coordinates are output. The least squares method np.linalg.lstsq(A, B, rcond=None)[0] is used to solve the transformation matrix from two dimensions to three dimensions, and the two-dimensional coordinates of the bolt holes are transformed into three-dimensional coordinates. The cv2.HoughCircles(edges, cv2.HOUGH_GRADIENT, 1, 50, param1=40, param2=20, minRadius=20, maxRadius=28) function is a function in the openCV library.

9. The three-dimensional point cloud virtual assembly method based on image recognition according to claim 1, characterized in that: In step S5, the compute_transformation_matrix(source_points, target_points) function is constructed. The point cloud is converted into a Points object of skspatial. The rotation matrix is calculated through U, S, Vt = np.linalg.svd(covariance_matrix), R = Vt.T @ U.T. The translation variable is calculated through t = centroid_target - R @ centroid_source. The translation and rotation matrices are combined into a homogeneous transformation matrix through transformation_matrix = np.eye(4), transformation_matrix[:3, :3] = R, transformation_matrix[:3, 3] = t. This matrix is called to perform coordinate transformation on the two pieces of point cloud to obtain the assembled finished product.

Images