An identification and ranging method for engineering vehicles based on point cloud and image fusion

Abstract

The application discloses an engineering vehicle recognition and ranging method based on point cloud and image fusion. The method comprises the following steps: determining a vehicle reference object, and acquiring multi-sensor coordinate data of the vehicle reference object; determining a multi-sensor coordinate system conversion relationship; acquiring image data of a region where the vehicle is located, and performing distance recognition according to the image data to obtain an image recognition and ranging result; acquiring a point cloud recognition and ranging result according to the multi-sensor coordinate system conversion relationship and the image data of the region where the vehicle is located; and determining a final recognition and ranging result according to the image recognition result and the point cloud recognition result. The application improves the accuracy of the final recognition and ranging result by fusing the image recognition and ranging result and the point cloud recognition and ranging result, and improves the real-time performance of the point cloud recognition and ranging result and reduces the false detection rate of the point cloud recognition and ranging result by using the multi-sensor coordinate conversion relationship to preliminarily screen the point cloud data.

Description

Technical Field

[0001] This invention relates to the field of vehicle identification and ranging, and specifically to a method for identifying and ranging engineering vehicles based on point cloud and image fusion. Background Technology

[0002] With the rapid development of LiDAR sensors, the use of point cloud recognition technology for vehicle identification and ranging has attracted increasing attention. Existing methods first require segmenting the point cloud of the area where the vehicle is located, and then identifying the category of the objects to which the point cloud belongs. The accuracy of segmentation affects the accuracy of recognition; if the points to which objects belong cannot be correctly identified, recognition is impossible. Furthermore, even with correct point cloud segmentation, issues such as missing or sparse points exist, and significant biases can occur with distant point clouds.

[0003] Currently, numerous methods exist for accurately identifying and segmenting point clouds in areas where vehicles are located, such as voxel-based point cloud recognition and high-speed point cloud recognition of cylinders. Additionally, there are methods that project point clouds into bird's-eye views and then use image recognition to identify the vehicle's location for distance measurement. However, limited by the sensor's inherent accuracy, LiDAR can only acquire point cloud data up to 70 meters, and the point cloud data capable of accurately identifying the complete outline is often less than 50 meters, thus requiring further improvement in the accuracy of vehicle identification and distance measurement. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention provides a method for identifying and measuring the distance of engineering vehicles based on point cloud and image fusion. By fusing image recognition and distance measurement results with point cloud recognition and distance measurement results, the accuracy of the final identification and distance measurement results is improved. Furthermore, by utilizing multi-sensor coordinate transformation relationships to reduce and initially screen point cloud data, the real-time performance of point cloud recognition and distance measurement results is improved, and the false detection rate of point cloud recognition and distance measurement results is reduced.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0006] A method for identifying and ranging engineering vehicles based on point cloud and image fusion includes the following steps:

[0007] S1. Determine the vehicle reference object and acquire the multi-sensor coordinate data of the vehicle reference object;

[0008] S2. Based on the multi-sensor coordinate data of the vehicle reference object in step S1, determine the multi-sensor coordinate system transformation relationship;

[0009] S3. Acquire image data of the area where the vehicle is located, and perform distance recognition based on the image data of the area where the vehicle is located to obtain the image recognition distance measurement result;

[0010] S4. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the point cloud recognition ranging result.

[0011] S5. Based on the image recognition ranging results in step S3 and the point cloud recognition ranging results in step S4, determine the final recognition ranging result.

[0012] Furthermore, step S2 includes the following sub-steps:

[0013] S21. Calculate the variance of the multi-sensor coordinate data based on the multi-sensor coordinate data of the vehicle reference object in step S1.

[0014] S22. Based on the variance of the multi-sensor coordinate data in step S21, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system.

[0015] S23. Determine the transformation relationship of the multi-sensor coordinate system based on the rotation matrix and translation matrix of the multi-sensor coordinate system in step S22.

[0016] Further, in step S21, the variance of the multi-sensor coordinate data is calculated, expressed as:

[0017]

[0018] Where: H is the variance of the multi-sensor coordinate data, j is the data point index of the multi-sensor coordinate data, and N is the total number of data points of the multi-sensor coordinate data. Let C be the j-th data point of the first sensor coordinate data. A The centroid coordinates of the first sensor coordinate data. Let C be the j-th data point of the second sensor coordinate data. B Let T be the centroid coordinates of the second sensor coordinate data, and T be the translation matrix of the multi-sensor coordinate system.

[0019] Furthermore, step S22 includes the following sub-steps:

[0020] S221. Perform singular value decomposition on the variance of the multi-sensor coordinate data in sub-step S21 to obtain the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data.

[0021] S222. Based on the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data in step S221, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system.

[0022] Furthermore, in step S222, singular value decomposition is performed on the variance of the multi-sensor coordinate data to obtain the first orthogonal matrix and the second orthogonal matrix, represented as:

[0023] [U, S, V] = SVD(H)

[0024] Where: U is the first orthogonal matrix of the multi-sensor coordinate data, S is the diagonal matrix of the multi-sensor coordinate data, V is the second orthogonal matrix of the multi-sensor coordinate data, SVD is the singular value decomposition algorithm, and H is the variance of the multi-sensor coordinate data.

[0025] Further, in step S222, the rotation matrix of the multi-sensor coordinate system is calculated, and expressed as:

[0026] R = VU T

[0027] Where: R is the rotation matrix of the multi-sensor coordinate system, V is the second orthogonal matrix of the multi-sensor coordinate data, U is the first orthogonal matrix of the multi-sensor coordinate data, and T is the translation matrix of the multi-sensor coordinate system.

[0028] Further, in step S222, the translation matrix of the multi-sensor coordinate system is calculated, and expressed as:

[0029] T = -R*C A +C B

[0030] Where: T is the translation matrix of the multi-sensor coordinate system, R is the rotation matrix of the multi-sensor coordinate system, and C... A Let C be the centroid coordinates of the first sensor coordinate data. B The centroid coordinates of the second sensor coordinate data.

[0031] Furthermore, step S4 includes the following sub-steps:

[0032] S41. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the three-dimensional point cloud of the vehicle's location area.

[0033] S42. Perform preprocessing and clustering segmentation on the 3D point cloud of the vehicle's location area in sub-step S41 to obtain discrete point cloud clusters of the vehicle's location area.

[0034] S43. Perform point cloud recognition on the discrete point cloud clusters in the area where the vehicle is located in sub-step S42, and obtain the point cloud recognition ranging results.

[0035] Further, in step S5, determining the final identification distance measurement result specifically involves: comparing the image identification distance measurement result and the point cloud identification distance measurement result to determine the vehicle type; selecting the corresponding calculation method based on the vehicle type to obtain the distance between the vehicle of that type and the high-voltage line; and determining the final identification distance measurement result based on the distance between the vehicle and the high-voltage line.

[0036] The beneficial effects of this invention are as follows:

[0037] (1) This invention improves the accuracy of the final recognition and ranging result by fusing image recognition ranging results and point cloud recognition ranging results;

[0038] (2) This invention utilizes the coordinate transformation relationship of multiple sensors to reduce and initially screen point cloud data, thereby improving the real-time performance of point cloud recognition and ranging results and reducing the false detection rate of point cloud recognition and ranging results.

[0039] (3) The present invention determines the multi-sensor coordinate system transformation relationship by determining the vehicle reference object and using the multi-sensor coordinate data of the vehicle reference object. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of a method for identifying and ranging engineering vehicles based on point cloud and image fusion.

[0041] Figure 2 This is a schematic diagram of the YOLO network detection principle.

[0042] Figure 3 This is a schematic diagram of the relationship between a two-dimensional image and a point cloud view frustum. Detailed Implementation

[0043] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0044] like Figure 1 As shown, a method for identifying and ranging engineering vehicles based on point cloud and image fusion includes steps S1-S5, as detailed below:

[0045] S1. Determine the vehicle reference object and acquire the multi-sensor coordinate data of the vehicle reference object.

[0046] In an optional embodiment of the present invention, when determining the vehicle reference object, the present invention determines the reference object with a clear outline as the vehicle reference object, and uses multiple sensors to obtain the multi-sensor coordinate data of the reference object.

[0047] S2. Based on the multi-sensor coordinate data of the vehicle reference object in step S1, determine the multi-sensor coordinate system transformation relationship.

[0048] In an optional embodiment of the present invention, the present invention first performs unification between three-dimensional coordinate systems, and then determines the multi-sensor coordinate system transformation relationship based on the multi-sensor coordinate data of the vehicle reference object.

[0049] Step S2 includes the following sub-steps:

[0050] S21. Calculate the variance of the multi-sensor coordinate data based on the multi-sensor coordinate data of the vehicle reference object in step S1.

[0051] This invention calculates the variance of multi-sensor coordinate data, expressed as:

[0052]

[0053] Where: H is the variance of the multi-sensor coordinate data, j is the data point index of the multi-sensor coordinate data, and N is the total number of data points of the multi-sensor coordinate data. Let C be the j-th data point of the first sensor coordinate data. A The centroid coordinates of the first sensor coordinate data. Let C be the j-th data point of the second sensor coordinate data. B Let T be the centroid coordinates of the second sensor coordinate data, and T be the translation matrix of the multi-sensor coordinate system.

[0054] S22. Based on the variance of the multi-sensor coordinate data in step S21, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system.

[0055] Step S22 includes the following sub-steps:

[0056] S221. Perform singular value decomposition on the variance of the multi-sensor coordinate data in step S21 to obtain the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data.

[0057] This invention performs singular value decomposition on the variance of multi-sensor coordinate data to obtain a first orthogonal matrix and a second orthogonal matrix, represented as:

[0058] [U, S, V] = SVD(H)

[0059] Where: U is the first orthogonal matrix of the multi-sensor coordinate data, S is the diagonal matrix of the multi-sensor coordinate data, V is the second orthogonal matrix of the multi-sensor coordinate data, SVD is the singular value decomposition algorithm, and H is the variance of the multi-sensor coordinate data.

[0060] S222. Based on the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data in step S221, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system.

[0061] The rotation matrix of the multi-sensor coordinate system calculated in this invention is expressed as:

[0062] R = VU T

[0063] Where: R is the rotation matrix of the multi-sensor coordinate system, V is the second orthogonal matrix of the multi-sensor coordinate data, U is the first orthogonal matrix of the multi-sensor coordinate data, and T is the translation matrix of the multi-sensor coordinate system.

[0064] The translation matrix of the multi-sensor coordinate system calculated in this invention is expressed as:

[0065] T = -R*C A +C B

[0066] Where: T is the translation matrix of the multi-sensor coordinate system, R is the rotation matrix of the multi-sensor coordinate system, and C... A Let C be the centroid coordinates of the first sensor coordinate data. B The centroid coordinates of the second sensor coordinate data.

[0067] S23. Determine the transformation relationship of the multi-sensor coordinate system based on the rotation matrix and translation matrix of the multi-sensor coordinate system in step S22.

[0068] Specifically, the multi-sensor coordinate system transformation relationship determined in this invention refers to the transformation relationship between lidar and laser rangefinder. Due to distance limitations, lidar often struggles to directly measure the point cloud of high-voltage power lines. Therefore, this invention uses a laser rangefinder to measure the coordinates of the two endpoints and the lowest point of the overhang of the high-voltage power line, substitutes these coordinates into the catenary equation in the database, obtains the three-dimensional points of the high-voltage power line in the laser rangefinder, and then uses the transformation relationship to convert them into lidar point clouds. The transformation relationship between the laser rangefinder and lidar is solved using the PnP algorithm.

[0069] S3. Acquire image data of the area where the vehicle is located, and perform distance recognition based on the image data of the area where the vehicle is located to obtain the image recognition distance measurement result.

[0070] In an optional embodiment of the present invention, the present invention extracts vehicles and their locations from a two-dimensional color image using a YOLO network to obtain image data of the vehicle's location area. The YOLO network uses convolutional layers to extract image features and fully connected layers to predict the image location and category probability values. Therefore, YOLO only needs to perform one inference to obtain the confidence probability of all locations and their corresponding labels in the image, while networks such as RCNN require two steps: first solving for the object category, and then using regression to solve for object classification. Because of the reduced computational steps, the YOLO network can run at extremely high speeds, meeting real-time requirements. Figure 2 As shown, the YOLO network divides the input image into multiple grids, with each grid responsible for detecting objects falling into it. If the center coordinates of an object fall into a grid, then that grid is responsible for detecting the object. Figure 2The center point of the dog object falls within the cell in row 5, column 2, so this cell is responsible for predicting the dog object in the image. Each cell outputs B boundary range information and C probability information of the object belonging to a certain category. The boundary range information contains 5 data values: x, y, w, h, and confidence score. x represents the x-coordinate of the center position of the predicted object's boundary range, y represents the y-coordinate of the center position of the predicted object's boundary range, w is the width of the boundary range, and h is the height of the boundary range. The confidence score reflects whether the current boundary range contains the object and the accuracy of the object's position.

[0071] The loss function of the YOLO network is expressed as:

[0072]

[0073] Where: loss is the loss function of the YOLO network, i is the grid number into which the input image is segmented, S is the number of rows of the grid into which the input image is segmented, coordError is the location correlation error, iouError is the intersection-union ratio error between grids containing objects and grids not containing objects, and classError is the classification error.

[0074] This invention demonstrates, through practical results, that the YOLO network exhibits faster detection speed and lower false positive rate for background images. The YOLO network treats object detection as a regression problem, resulting in a simple network structure that achieves a detection speed of 45fps while maintaining accuracy. During training and inference, the YOLO network detects the overall information of the entire image, whereas bounding box-based object detection methods can only detect local image information within candidate boxes. Therefore, if data from the image background (not objects) is included in a candidate box and fed into the detection network, it is easily misdetected as an object. Tests show that the YOLO network's false positive rate for background images is less than half that of Fast R-CNN (a bounding box-based object detection method).

[0075] S4. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the point cloud recognition ranging result.

[0076] In an optional embodiment of the present invention, the present invention obtains a three-dimensional point cloud of the vehicle's location based on the multi-sensor coordinate system transformation relationship and the image data of the vehicle's location area, performs preprocessing and clustering segmentation on the three-dimensional point cloud of the vehicle's location area to obtain discrete point cloud clusters of the vehicle's location area, and performs point cloud recognition on the discrete point cloud clusters of the vehicle's location area to obtain point cloud recognition ranging results.

[0077] Step S4 includes the following sub-steps:

[0078] S41. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the three-dimensional point cloud of the vehicle's location area.

[0079] Specifically, this invention places the boundary ranges of the same label in the image data of the vehicle's location into a set, and then projects the point cloud onto the image data of the vehicle's location. This invention compares the projected position of the point cloud with the set range; if a point belongs to that set, it is placed into the point cloud set for that label. This invention calculates the sets to which all point clouds belong to remove points not in any set, i.e., a large amount of useless point cloud information such as roads, walls, and streetlights, extracting only the point cloud that may contain vehicles within its label range. Figure 3 As shown, this invention obtains a point cloud view cone that may contain vehicles through image frames. This method reduces the computational load of the point cloud recognition network and also minimizes interference from irrelevant point clouds in the recognition process.

[0080] S42. Perform preprocessing and clustering segmentation on the 3D point cloud of the area where the vehicle is located in step S41 to obtain discrete point cloud clusters of the area where the vehicle is located.

[0081] Specifically, for point clouds within the view frustum, this invention uses image bounding boxes instead of pixel-level segmentation. These bounding boxes include the ground near the vehicle, trees in the background, and buildings. Furthermore, since the point clouds obtained by LiDAR contain noise, preprocessing of the 3D point cloud in the vehicle's location is necessary. Specifically, this invention employs statistical filters to remove outliers and noise to preprocess the 3D point cloud in the vehicle's location. Simultaneously, DBSCAN clustering is used to transform point clouds meeting density requirements into point cloud clusters, achieving clustering and segmentation of the 3D point cloud in the vehicle's location, ultimately further reducing the computational load of the point cloud network.

[0082] S43. Perform point cloud recognition on the discrete point cloud clusters in the area where the vehicle is located in sub-step S42, and obtain the point cloud recognition ranging results.

[0083] Specifically, this invention utilizes a point cloud recognition network to perform point cloud recognition on discrete point cloud clusters in the area where the vehicle is located. The point cloud recognition network employs a PV-RCNN neural network. PV-RCNN is a novel network structure that combines point-based and voxel-based approaches. Voxel-based networks are efficient but suffer from information loss, reducing the fine-grained accuracy of localization; point-based networks have high computational costs but offer a larger receptive field of view. The PV-RCNN neural network averages some grid points within each 3D candidate box, then uses the farthest point sampling method to obtain the surrounding points of that grid point, and further refines the results to obtain the final candidate box. The PV-RCNN neural network consists of two parts: voxel-to-keypoint encoding and keypoint-to-grid point feature extraction. Voxel-to-keypoint encoding first uses a voxel method to extract features from the entire scene, while simultaneously having one branch sample the scene using the farthest point, then retrieving multi-scale voxel features. In this way, only voxel features are actually used, but they are reflected in the keypoint features. Keypoint-to-grid point feature extraction fuses the keypoint features from the first step with the features of the grid points of interest.

[0084] S5. Based on the image recognition ranging results in step S3 and the point cloud recognition ranging results in step S4, determine the final recognition ranging result.

[0085] In an optional embodiment of the present invention, the present invention compares the image recognition result and the point cloud recognition result to determine the vehicle type, selects the corresponding calculation method according to the vehicle type to obtain the distance between the vehicle of that type and the high-voltage line, and determines the final recognition distance measurement result based on the distance between the vehicle and the high-voltage line.

[0086] Specifically, this invention compares the point cloud recognition ranging result (vehicle type identified by the point cloud) with the image recognition ranging result (image recognition ranging result obtained by the YOLO network). If both results identify the same vehicle type, the vehicle type is determined. If the image recognition ranging result and the point cloud recognition ranging result do not match, this invention will make a judgment based on the detected vehicle shape, distance, and other information, using a misidentification database obtained from actual road tests. After determining the vehicle type, this invention selects a corresponding preset method to calculate the distance between the vehicle and the obstacle based on the vehicle type in the database, obtaining the optimal speed and accuracy. For ordinary vehicles, this invention only needs to subtract a preset roof height from the calculated vehicle's center of gravity position to obtain the distance between this type of vehicle and the high-voltage line, and determine the final recognition ranging result based on the distance between the vehicle and the high-voltage line. For tower crane-type vehicles, this invention needs to accurately calculate the distance between the highest point and the high-voltage line and determine the final recognition ranging result based on the distance between the vehicle and the high-voltage line. This invention can quickly determine whether the vehicle will cause damage to the high-voltage line based on the final recognition ranging result.

[0087] On the same computer, using the same set of collected data, this invention achieved an average frame rate of 18 frames per second (compared to 11 frames per second for traditional algorithms), a 63.6% improvement in running speed, and a recognition accuracy of 87.23% (67.97% accuracy for single point cloud recognition). The improvement in recognition accuracy is specifically due to the following: This invention utilizes image recognition results, ensuring clear identification of distant objects in the image. Therefore, when used as a precondition input into the point cloud recognition network, it enables the recognition of point clouds that were previously difficult to identify. Furthermore, in the image data of the vehicle's location, this invention successfully segments the vehicle's point cloud into different candidate bounding boxes, thus preventing point clouds of different objects from becoming stuck together.

[0088] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for identifying and ranging engineering vehicles based on point cloud and image fusion, characterized in that, Includes the following steps: S1. Determine the vehicle reference object and acquire the multi-sensor coordinate data of the vehicle reference object; the multi-sensor includes lidar and laser rangefinder; S2. Based on the multi-sensor coordinate data of the vehicle reference object in step S1, determine the multi-sensor coordinate system transformation relationship; the multi-sensor coordinate system transformation relationship is the transformation relationship between the lidar and the laser rangefinder; S3. Acquire image data of the area where the vehicle is located, and perform distance recognition based on the image data of the area where the vehicle is located to obtain the image recognition distance measurement result; S4. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the point cloud recognition ranging result. S5. Determine the final recognition and ranging result based on the image recognition ranging result in step S3 and the point cloud recognition ranging result in step S4. Step S4 includes the following sub-steps: S41. Based on the multi-sensor coordinate system transformation relationship in step S2 and the image data of the vehicle's location area in step S3, obtain the three-dimensional point cloud of the vehicle's location area. S42. Statistical filters are used to remove outliers and noise to preprocess the 3D point cloud of the vehicle's location. DBSCAN clustering is used to transform the point cloud that meets the density requirements into point cloud clusters to achieve clustering and segmentation of the 3D point cloud of the vehicle's location, thereby obtaining discrete point cloud clusters of the vehicle's location. S43. Perform point cloud recognition on the discrete point cloud clusters of the area where the vehicle is located in sub-step S42, and obtain the point cloud recognition ranging results. In step S5, determining the final identification and ranging result specifically involves: measuring the coordinates of the two ends of the high-voltage line and the lowest point of the suspension using a laser rangefinder, substituting them into the catenary equation in the database to obtain the three-dimensional points of the high-voltage line in the laser rangefinder, and then using the multi-sensor coordinate system transformation relationship to convert the three-dimensional points of the high-voltage line into a lidar point cloud; comparing the image recognition ranging result and the point cloud recognition ranging result to determine the vehicle type: selecting the corresponding calculation method according to the vehicle type to obtain the distance between the vehicle and the high-voltage line; for ordinary vehicles, subtracting the preset roof height after calculating the vehicle's center of gravity to obtain the distance between the vehicle and the high-voltage line; for tower crane type vehicles, accurately calculating the distance between the highest point and the high-voltage line; and determining the final identification and ranging result based on the distance between the vehicle and the high-voltage line.

2. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 1, characterized in that, Step S2 includes the following sub-steps: S21. Calculate the variance of the multi-sensor coordinate data based on the multi-sensor coordinate data of the vehicle reference object in step S1. S22. Based on the variance of the multi-sensor coordinate data in step S21, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system. S23. Determine the transformation relationship of the multi-sensor coordinate system based on the rotation matrix and translation matrix of the multi-sensor coordinate system in step S22.

3. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 2, characterized in that, In step S21, the variance of the multi-sensor coordinate data is calculated and expressed as: in: The variance of the multi-sensor coordinate data, The data point sequence number of the multi-sensor coordinate data. The total number of data points in the multi-sensor coordinate data. The first sensor coordinate data Data points, The centroid coordinates of the first sensor coordinate data. The second sensor coordinate data Data points, The centroid coordinates of the second sensor coordinate data. This is the translation matrix of the multi-sensor coordinate system.

4. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 2, characterized in that, Step S22 includes the following sub-steps: S221. Perform singular value decomposition on the variance of the multi-sensor coordinate data in sub-step S21 to obtain the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data. S222. Based on the first orthogonal matrix and the second orthogonal matrix of the multi-sensor coordinate data in step S221, calculate the rotation matrix and translation matrix of the multi-sensor coordinate system.

5. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 4, characterized in that, In step S222, singular value decomposition is performed on the variance of the multi-sensor coordinate data to obtain the first orthogonal matrix and the second orthogonal matrix, as follows: in: This is the first orthogonal matrix of the multi-sensor coordinate data. This is a diagonal matrix of multi-sensor coordinate data. This is the second orthogonal matrix of the multi-sensor coordinate data. This is a singular value decomposition algorithm. The variance of the multi-sensor coordinate data.

6. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 5, characterized in that, In step S222, the rotation matrix of the multi-sensor coordinate system is calculated, and is expressed as: in: The rotation matrix of the multi-sensor coordinate system. This is the second orthogonal matrix of the multi-sensor coordinate data. This is the first orthogonal matrix of the multi-sensor coordinate data. This is the translation matrix of the multi-sensor coordinate system.

7. The method for identifying and ranging engineering vehicles based on point cloud and image fusion according to claim 6, characterized in that, In step S222, the translation matrix of the multi-sensor coordinate system is calculated, and expressed as: in: The translation matrix of the multi-sensor coordinate system. The rotation matrix of the multi-sensor coordinate system. The centroid coordinates of the first sensor coordinate data. The centroid coordinates of the second sensor coordinate data.

Images