A vehicle dimension measurement method and system based on a mobile acquisition device

Abstract

This invention discloses a method and system for measuring vehicle dimensions based on a mobile acquisition device. The method includes: time synchronization and joint calibration of a lidar and a camera; obtaining the target vehicle position as the target detection result; stitching together each frame of point cloud data to complete the three-dimensional reconstruction of the current space and reproduce the vehicle outline point cloud information; segmenting to obtain ground point cloud and non-ground point cloud; combining the target detection result to obtain the target point cloud; extracting vehicle outline points from the target point cloud; fitting the length L and width W of the target vehicle's outline dimensions based on the yaw angle of the vehicle outline points; obtaining the height H of the target vehicle's outline dimensions. The system includes: a mobile acquisition device, a data preprocessing module, a target detection module, a point cloud stitching module, a ground point segmentation module, a target point cloud segmentation module, a vehicle outline point segmentation module, a first dimension calculation module, and a second dimension calculation module.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for measuring the dimensions of a car based on a mobile acquisition device. Background Technology

[0002] Measuring vehicle dimensions is crucial to ensuring compliance with laws, regulations, and safety standards. Different countries have varying vehicle size restrictions, including regulations on length, width, and height, to ensure safe driving on roads and avoid collisions with other vehicles or road infrastructure. Traditional vehicle dimension measurement is typically manual, time-consuming, and labor-intensive. In recent years, automated measurement technologies based on point cloud analysis of vehicle dimensions have emerged. However, these methods mostly require fixed instrument positions, resulting in poor flexibility, slow operation, and complex post-processing. Furthermore, they cannot simultaneously measure the dimensions of multiple vehicles. Summary of the Invention

[0003] In view of this, the present invention provides a method and system for measuring automobile dimensions based on a mobile acquisition device, which at least solves the problem of poor flexibility in existing methods for measuring external dimensions.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for measuring vehicle dimensions based on a mobile data acquisition device, wherein the mobile data acquisition device is equipped with a camera and a lidar for acquiring data, includes the following steps:

[0006] S1. The lidar and camera undergo time synchronization and joint calibration;

[0007] S2. The trained target detection model is used to detect targets in the image data collected by the camera, and the position of the target vehicle is obtained as the target detection result.

[0008] S3. Control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform the point cloud of each frame during the movement to the coordinate system of the first frame according to the rotation transformation matrix. Then, stitch together the point cloud data of each frame to obtain a global map to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline.

[0009] S4. Segment the global map to obtain ground point cloud and non-ground point cloud;

[0010] S5. Combining the target detection results obtained in S2, segment the point cloud of the target vehicle in the non-ground point cloud to obtain the target point cloud;

[0011] S6. Extract the vehicle contour points from the target point cloud;

[0012] S7. Fit the length L and width W of the target vehicle's outline dimensions based on the yaw angle of the vehicle's outline points, and obtain the center point of the vehicle outline on the xoy plane.

[0013] S8. Traverse the target point cloud obtained in S5 to obtain its maximum value Zmax on the Z-axis. Combine the ground point cloud segmented in S4 and the center point of the car contour on the xoy plane obtained in S7 to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, obtain the height H in the outer dimensions of the target vehicle.

[0014] Preferably, the specific content of S1 includes:

[0015] Data collected by the camera and LiDAR nodes on the mobile acquisition device is synchronized in time. Furthermore, by acquiring checkerboard-patterned images and point cloud data from different locations, the intrinsic parameters Pr of the camera and the extrinsic parameters Tr of the LiDAR and camera are calibrated. Pr and Tr are:

[0016]

[0017]

[0018] In the formula, f x and f y Let be the focal lengths of the camera coordinate system x and y axes, u0 and v0 be the actual positions of the principal points on the x and y axes, R be the rotation matrix from the lidar to the camera, and T be the translation matrix from the lidar to the camera.

[0019] Preferably, the specific content of S3 includes:

[0020] S31. Transfer the point cloud from the previous frame S c ={a0,...,a N} and the point cloud T of the next frame c ={b0,...,b N The surface containing the point is modeled as a Gaussian distribution, such that it satisfies... and

[0021] S32. Initialize the rotation transformation matrix, set the point cloud resolution, and transform the point cloud T... c Voxelization is performed, and the mean μ, variance C, and number of points N in each voxel are recorded.

[0022] S33. Regarding S c The point cloud is traversed, and the voxel position index of the current point is calculated based on the point cloud resolution. If it is in T c If the value is not found in the function, skip the current point and calculate the error E based on the optimization function F. iand Hessian matrix H i The process is then accumulated, and after traversal, a Newton-Gaussian iteration is performed. If the rotation transformation matrix does not converge, step S33 is repeated. The optimization function is:

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] In the formula, T is the translation matrix from the lidar to the camera, and N... i For a i Nearby T c The number of point clouds in a voxel, J i Let Ta be a Jacobian matrix. i |x l Point a i Convert to point cloud T c The value of the x-axis in the coordinate system; Ta i |y l Point a i Convert to point cloud T c The value of the y-axis in the coordinate system;

[0029] S34. Transform the point cloud at each time step to the coordinate system of the first frame, where the transformation matrix from each frame to the first frame is Rt. n =(RT) t0 ×RT t1 ×....×RT tn ) -1 Once the mobile data acquisition device stops moving, the global map stitching is complete, and the car outlines are reconstructed simultaneously.

[0030] Preferably, the specific content of S4 includes:

[0031] S41. Project the unordered point cloud data onto the xoy plane to transform it into an ordered data representation. The xoy plane is considered as a circle with an infinite radius. Divide the circle (with radii greater than r_min and less than r_max) into n_segments of sectors. Within each sector, divide it into n_bins of rings based on distance. In each ring, transform the point cloud {x,y,z} into {d,z} to achieve dimensionality reduction. The dimensionality reduction process is as follows:

[0032]

[0033] S42. Traverse each sector, starting from the first sector ring to fit a straight line. Use the maximum slope of the line (max_slope), the fitting error of the line (max_error_square), and the intercept of the line on the z-axis to select points to add to the line. Use the longest distance threshold of the line (long_threshold) to determine whether to start fitting a new line, until the line fitting in each sector is completed.

[0034] S43. Traverse the point cloud. If the distance from a point to a line is less than the set maximum distance max_dist_to_line, it is considered a ground point. Based on the correspondence between the reduced point cloud and the original point cloud, the ground points and non-ground points in the original point cloud are deduced.

[0035] S44. Calculate the centroid {x} of the ground point. g ,y g ,z g}, set ground_threshhold, and further filter out z-values ​​from non-ground point clouds using a pass-through filter. g Point cloud below the +ground_threshhold height.

[0036] Preferably, the specific content of S5 includes:

[0037] S51. Transform the point cloud from the lidar coordinate system to the pixel coordinate system. If the point cloud falls within the two-dimensional bounding box of the target detection result obtained in S2, it is determined to be a point to be clustered. Each two-dimensional detection box in the image has a corresponding cluster of point clouds to be clustered.

[0038] S52. Cluster each point cloud cluster to be clustered to obtain several point cloud clusters. Project the centroid of the point cloud cluster onto the image and select the point cloud cluster with the smallest centroid distance from the center of the two-dimensional detection box as the target point cloud.

[0039] S53. Construct the target point cloud into an ocTree tree with a resolution of tree_resolution. Traverse the target point cloud and perform a spherical neighborhood search with radius radius for each point in the ocTree tree. If the number of neighborhoods is less than erode_count, it is judged as a noise point. Finally, remove all noise points from the target point cloud.

[0040] Preferably, the specific content of the point cloud transformation from the lidar coordinate system to the pixel coordinate system in S51 includes:

[0041] Let the coordinates of any point P in the lidar coordinate system be (P x ,P y ,P z ), whose coordinates in the camera coordinate system are (X w ,Y w Zw Given a point P with coordinates (x, y) in the pixel coordinate system, the transformation relationship from the lidar coordinate system to the pixel coordinate system is as follows:

[0042]

[0043]

[0044] Where Pr is the intrinsic parameter of the camera, and Tr is the extrinsic parameter of the lidar to the camera.

[0045] Preferably, the specific content of S6 includes:

[0046] Project the target point cloud onto the xoy plane and use the ConvexHull convex hull algorithm to find the contour points of the point cloud. Traverse all contour points, build an Octree tree to find the point cloud within a spherical region of radius radius, calculate the normal vector, and if the angle between the normal vector of the point cloud and the normal vector of the contour exceeds contour_deg, then the point is determined to be a convex part.

[0047] All points identified as protruding parts are filtered out from the target point cloud. After erosion and denoising of the remaining target point cloud, the ConvexHull convex hull algorithm is used again to find the contour points of the point cloud.

[0048] Preferably, the specific content of S7 includes:

[0049] Traverse the range 0 to π / 2 with a step size of reasearch_deg, rotate the contour point cloud by an angle of -θ, and obtain the values ​​in two dimensions, C1 and C2:

[0050]

[0051] In the formula, θ = n × research_deg, and Q is the set of contour points {(x i ,y i ,z i )|i=0,1,2,…,m}, where m is the number of contour points;

[0052] Based on the minimum area criterion, the loss value is recorded for each step length. The loss function is:

[0053] Loss=(max(C1)-min(C1))×(max(C2)-min(C2)) (12)

[0054] After the traversal is completed, the yaw angle θ corresponding to the minimum loss is selected. Then the corresponding length L is max(C1)-min(C1), the width W is max(C2)-min(C2), the center point cx on the X-axis is (max(C1)+min(C1)) / 2, and the center point cy on the Y-axis is (max(C2)+min(C2)) / 2.

[0055] Preferably, the specific content of S8 includes:

[0056] The target point cloud obtained by S5 is traversed to obtain its maximum value Zmax on the Z-axis;

[0057] A KD-tree is constructed from the ground points obtained in S4. Based on the center point on the xoy plane obtained in S7, the nearest neighbor search algorithm is used to find n_neighbors of its nearest point cloud. The ground plane around the car is fitted based on the found point cloud, where the ground plane direction is:

[0058] z = ax + by + c (13)

[0059] In the formula, a, b, and c are the coefficients of the plane. According to the principle of least squares fitting, the sum of the residuals of the plane fitting is:

[0060]

[0061] When E takes its minimum value but:

[0062]

[0063] Solving the above equation yields the plane equation coefficients a, b, and c, from which the height H of the car can be calculated:

[0064] H=Zmax-(a×cx+b×cy+c) (16).

[0065] A vehicle size measurement system based on a mobile acquisition device includes: a mobile acquisition device, a data preprocessing module, a target detection module, a point cloud stitching module, a ground point segmentation module, a target point cloud segmentation module, a vehicle contour point segmentation module, a first size calculation module, and a second size calculation module.

[0066] The mobile data acquisition device is equipped with a lidar and a camera for data collection.

[0067] The data preprocessing module is used to perform time synchronization and joint calibration of the lidar and camera on the mobile acquisition device;

[0068] The target detection module is used to train the target detection model and then use the trained target detection model to detect targets in the image data collected by the camera, obtaining the target vehicle position as the target detection result.

[0069] The point cloud stitching module is used to control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform each frame of point cloud in the movement process to the coordinate system of the first frame according to the rotation transformation matrix. The point cloud data of each frame is stitched together to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline.

[0070] The ground point segmentation module is used to segment the ground point cloud into non-ground point clouds;

[0071] The target point cloud segmentation module is used to segment the point cloud of the target vehicle in the non-ground point cloud by combining the target detection results obtained by the target detection module, so as to obtain the target point cloud.

[0072] The vehicle contour point segmentation module is used to extract vehicle contour points from the target point cloud.

[0073] The first dimension calculation module is used to fit the length L and width W of the target vehicle's outer dimensions based on the yaw angle of the vehicle's outline points, and to obtain the center point of the vehicle's outline on the plane xoy.

[0074] The second dimension calculation module is used to traverse the target point cloud obtained by the target point cloud segmentation module to obtain its maximum value Zmax on the Z-axis. It combines the ground point cloud segmented by the ground point segmentation module and the center point of the car outline on the plane xoy obtained by the first dimension calculation module to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, the height H in the outer dimensions of the target vehicle is obtained.

[0075] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for measuring automobile dimensions based on a mobile acquisition device, which has the following beneficial effects:

[0076] This invention, by mounting a camera and LiDAR on a mobile acquisition device, offers advantages over existing methods, including automatic measurement, mobile detection, no need for floor space, no requirements on the relative orientation of the vehicle and the measuring device, high measurement accuracy, fast operation speed, and the ability to measure multiple vehicles simultaneously, thus broadening its application range. In the point cloud stitching section, this invention can reconstruct the outlines of multiple vehicles. Furthermore, by fusing with visual target detection, it can simultaneously calculate the dimensions of multiple vehicles. Because it incorporates vehicle yaw angle fitting, the sensor and the measured object do not need to be directly or sideways aligned, resulting in a large measurement range. Additionally, the algorithm relies on LiDAR and a camera, requiring only a low-cost embedded platform that can be mounted on a mobile platform, eliminating the limitation of fixed sensor installation, thus offering high flexibility and ease of portability. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 This invention provides an overall flowchart of a method for measuring automobile dimensions based on a mobile acquisition device.

[0079] Figure 2 A flowchart of ground point segmentation provided for embodiments of the present invention;

[0080] Figure 3 A flowchart of target point cloud segmentation provided in an embodiment of the present invention;

[0081] Figure 4 A flowchart for height calculation provided for embodiments of the present invention;

[0082] Figure 5 This is a flowchart of the target detection model training process provided in an embodiment of the present invention;

[0083] Figure 6 This is a schematic diagram of a single-target measurement result provided in an embodiment of the present invention;

[0084] Figure 7 This is a schematic diagram of multi-target measurement results provided in an embodiment of the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] This invention provides a method for measuring vehicle dimensions based on a mobile data acquisition device. The mobile data acquisition device is equipped with a camera and a lidar for acquiring data, such as... Figure 1 As shown, it includes the following steps:

[0087] S1. The lidar and camera undergo time synchronization and joint calibration;

[0088] S2. The trained target detection model is used to detect targets in the image data collected by the camera, and the position of the target vehicle is obtained as the target detection result.

[0089] S3. Control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform the point cloud of each frame during the movement to the coordinate system of the first frame according to the rotation transformation matrix. Then, stitch together the point cloud data of each frame to obtain a global map to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline.

[0090] S4. Segment the global map to obtain ground point cloud and non-ground point cloud;

[0091] S5. Combining the target detection results obtained in S2, segment the point cloud of the target vehicle in the non-ground point cloud to obtain the target point cloud;

[0092] S6. Extract the vehicle contour points from the target point cloud;

[0093] S7. Fit the length L and width W of the target vehicle's outline dimensions based on the yaw angle of the vehicle's outline points, and obtain the center point of the vehicle outline on the xoy plane.

[0094] S8. Traverse the target point cloud obtained in S5 to obtain its maximum value Zmax on the Z-axis. Combine the ground point cloud segmented in S4 and the center point of the car contour on the xoy plane obtained in S7 to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, obtain the height H in the outer dimensions of the target vehicle.

[0095] To further implement the above technical solution, the specific content of S1 includes:

[0096] Data collected by the camera and LiDAR nodes on the mobile acquisition device is synchronized in time. Furthermore, by acquiring checkerboard-patterned images and point cloud data from different locations, the intrinsic parameters Pr of the camera and the extrinsic parameters Tr of the LiDAR and camera are calibrated. Pr and Tr are:

[0097]

[0098]

[0099] In the formula, f x and f y Let be the focal lengths of the camera coordinate system x and y axes, u0 and v0 be the actual positions of the principal points on the x and y axes, R be the rotation matrix from the lidar to the camera, and T be the translation matrix from the lidar to the camera.

[0100] It should be noted that:

[0101] Since the camera does not support hardware synchronization, this embodiment uses software time synchronization, such as... Figure 3 As shown, data is collected in ROS via a camera and a LiDAR node, and the data is synchronized using nearest neighbor matching. After time synchronization, images and point cloud data of the checkerboard pattern at different locations are collected, and MATLAB toolboxes are used to calibrate the camera's intrinsic parameter Pr and the LiDAR's extrinsic parameter Tr.

[0102] In this embodiment, R is a 3×3 matrix and T is a 1×3 matrix.

[0103] To further implement the above technical solution, the specific content of S3 includes:

[0104] S31. Transfer the point cloud from the previous frame S c ={a0,...,a N} and the point cloud T of the next frame c ={b0,...,b N The surface containing the point is modeled as a Gaussian distribution, such that it satisfies... and

[0105] S32. Initialize the rotation transformation matrix, set the point cloud resolution, and transform the point cloud T... c Voxelization is performed, and the mean μ, variance C, and number of points N in each voxel are recorded.

[0106] S33. Regarding S c The point cloud is traversed, and the voxel position index of the current point is calculated based on the point cloud resolution. If it is in T c If the value is not found in the function, skip the current point and calculate the error E based on the optimization function F. iand Hessian matrix H i The process is then accumulated, and after traversal, a Newton-Gaussian iteration is performed. If the rotation transformation matrix does not converge, step S33 is repeated. The optimization function is:

[0107]

[0108]

[0109]

[0110]

[0111]

[0112] In the formula, T is the translation matrix from the lidar to the camera, and N... i For a i Nearby T c The number of point clouds in a voxel, J i Let Ta be a Jacobian matrix. i |x l Point a i Convert to point cloud T c The value of the x-axis in the coordinate system; Ta i |y l Point a i Convert to point cloud T c The value of the y-axis in the coordinate system;

[0113] S34. Transform the point cloud at each time step to the coordinate system of the first frame, where the transformation matrix from each frame to the first frame is Rt. n =(RT) t0 ×RT t1 ×....×RT tn ) -1 Once the mobile data acquisition device stops moving, the global map stitching is complete, and the car outlines are reconstructed simultaneously.

[0114] It should be noted that:

[0115] To reconstruct the complete outline of the car, this embodiment selects VGICP point cloud as the registration algorithm for two frames of point cloud.

[0116] To further implement the above technical solutions, such as Figure 2 As shown, the specific content of S4 includes:

[0117] S41. Project the unordered point cloud data onto the xoy plane to transform it into an ordered data representation. The xoy plane is considered as a circle with an infinite radius. Divide the circle (with radii greater than r_min and less than r_max) into n_segments of sectors. Within each sector, divide it into n_bins of rings based on distance. In each ring, transform the point cloud {x,y,z} into {d,z} to achieve dimensionality reduction. The dimensionality reduction process is as follows:

[0118]

[0119] S42. Traverse each sector, starting from the first sector ring to fit a straight line. Use the maximum slope of the line (max_slope), the fitting error of the line (max_error_square), and the intercept of the line on the z-axis to select points to add to the line. Use the longest distance threshold of the line (long_threshold) to determine whether to start fitting a new line, until the line fitting in each sector is completed.

[0120] S43. Traverse the point cloud. If the distance from a point to a line is less than the set maximum distance max_dist_to_line, it is considered a ground point. Based on the correspondence between the reduced point cloud and the original point cloud, the ground points and non-ground points in the original point cloud are deduced.

[0121] S44. Calculate the centroid {x} of the ground point. g ,y g ,z g}, set ground_threshhold, and further filter out z-values ​​from non-ground point clouds using a pass-through filter. g Point cloud below the +ground_threshhold height.

[0122] It should be noted that:

[0123] Before calculating the target volume, it is necessary to distinguish between ground points and non-ground points to avoid ground points affecting the subsequent 3D bounding box fitting. In this embodiment, the LineFit algorithm, which has the highest accuracy in ground segmentation, is selected to initially segment ground points and non-ground points. Then, the centroid is calculated based on the segmented ground points, and a pass-through filter is used to further filter out ground point clouds that have not been completely segmented.

[0124] To further implement the above technical solutions, such as Figure 3 As shown, the specific content of S5 includes:

[0125] S51. Transform the point cloud from the lidar coordinate system to the pixel coordinate system. If the point cloud falls within the two-dimensional bounding box of the target detection result obtained in S2, it is determined to be a point to be clustered. Each two-dimensional detection box in the image has a corresponding cluster of point clouds to be clustered.

[0126] S52. Cluster each point cloud cluster to be clustered to obtain several point cloud clusters. Project the centroid of the point cloud cluster onto the image and select the point cloud cluster with the smallest centroid distance from the center of the two-dimensional detection box as the target point cloud.

[0127] S53. Construct the target point cloud into an ocTree tree with a resolution of tree_resolution. Traverse the target point cloud and perform a spherical neighborhood search with radius radius for each point in the ocTree tree. If the number of neighborhoods is less than erode_count, it is judged as a noise point. Finally, remove all noise points from the target point cloud.

[0128] It should be noted that:

[0129] After obtaining the non-ground point cloud, it is necessary to extract the target point cloud. S5 uses visual projection combined with the target detection results to filter out most of the non-target point cloud, and then performs clustering and erosion denoising on the remaining point cloud to segment out the target point cloud.

[0130] In this embodiment, the specific method for clustering each point cloud cluster to be clustered in S52 is as follows:

[0131] 1) Determine the neighborhood radius r = 2Δver based on the center position of the point cloud cluster and the point cloud distribution characteristics of the lidar, and set the minimum number of points Minpts:

[0132]

[0133] In the formula, the value of the centroid of the point cloud to be clustered on the z-axis is z. d The horizontal distance to the lidar is d

[0134] 2) For each sample point in the point cloud cluster to be clustered, calculate the number of points in its r-neighborhood. If the number of points in the r-neighborhood of a point is more than Minpts, then mark the current point as the core point.

[0135] 3) For points that are not core points but fall within the r-neighborhood of core points, mark them as boundary points;

[0136] 4) Starting from any unvisited core point and the points in its r-neighborhood, form a new cluster. Repeat steps 1)-4) to iterate over the unvisited core points until all core points have been visited.

[0137] 5) Mark all points that are not core points and are not in any cluster as noise points.

[0138] To further implement the above technical solution, the specific details of transforming the point cloud from the lidar coordinate system to the pixel coordinate system in S51 include:

[0139] Let the coordinates of any point P in the lidar coordinate system be (P x ,P y ,P z ), whose coordinates in the camera coordinate system are (X w ,Y w Z w Given a point P with coordinates (x, y) in the pixel coordinate system, the transformation relationship from the lidar coordinate system to the pixel coordinate system is as follows:

[0140]

[0141]

[0142] Where Pr is the intrinsic parameter of the camera, and Tr is the extrinsic parameter of the lidar to the camera.

[0143] To further implement the above technical solution, the specific content of S6 includes:

[0144] Project the target point cloud onto the xoy plane and use the ConvexHull convex hull algorithm to find the contour points of the point cloud. Traverse all contour points, build an Octree tree to find the point cloud within a spherical region of radius radius, calculate the normal vector, and if the angle between the normal vector of the point cloud and the normal vector of the contour exceeds contour_deg, then the point is determined to be a convex part.

[0145] All points identified as protruding parts are filtered out from the target point cloud. After erosion and denoising of the remaining target point cloud, the ConvexHull convex hull algorithm is used again to find the contour points of the point cloud.

[0146] It should be noted that:

[0147] The specific steps of the ConvexHull convex hull algorithm are as follows: 1) Select the bottom leftmost point from the point set as the starting point and use it as part of the convex hull; 2) Sort the remaining points according to the tangent value of the polar angle (relative to the starting point); 3) Starting from the sorted point set, traverse each point in turn. For the current point, if it forms a right turn with a point in the constructed convex hull (i.e., the point is outside the convex hull), remove the previous point that constitutes the convex hull until a point is found that allows the convex hull to continue to maintain its convexity before adding the current point to the convex hull; 4) After all points have been traversed, the constructed convex hull can be obtained.

[0148] To further implement the above technical solution, the specific content of S7 includes:

[0149] Traverse the range 0 to π / 2 with a step size of reasearch_deg, rotate the contour point cloud by an angle of -θ, and obtain the values ​​in two dimensions, C1 and C2:

[0150]

[0151] In the formula, θ = n × research_deg, and Q is the set of contour points {(x i ,y i ,z i )|i=0,1,2,…,m}, where m is the number of contour points;

[0152] Based on the minimum area criterion, the loss value is recorded for each step length. The loss function is:

[0153] Loss=(max(C1)-min(C1))×(max(C2)-min(C2)) (12)

[0154] After the traversal is completed, the yaw angle θ corresponding to the minimum loss is selected. Then the corresponding length L is max(C1)-min(C1), the width W is max(C2)-min(C2), the center point cx on the X-axis is (max(C1)+min(C1)) / 2, and the center point cy on the Y-axis is (max(C2)+min(C2)) / 2.

[0155] To further implement the above technical solutions, such as Figure 4 As shown, the specific content of S8 includes:

[0156] The target point cloud obtained by S5 is traversed to obtain its maximum value Zmax on the Z-axis;

[0157] A KD-tree is constructed from the ground points obtained in S4. Based on the center point on the xoy plane obtained in S7, the nearest neighbor search algorithm is used to find n_neighbors of its nearest point cloud. The ground plane around the car is fitted based on the found point cloud, where the ground plane direction is:

[0158] z = ax + by + c (13)

[0159] In the formula, a, b, and c are the coefficients of the plane. According to the principle of least squares fitting, the sum of the residuals of the plane fitting is:

[0160]

[0161] When E takes its minimum value but:

[0162]

[0163] Solving the above equation yields the plane equation coefficients a, b, and c, from which the height H of the car can be calculated:

[0164] H=Zmax-(a×cx+b×cy+c) (16).

[0165] A vehicle size measurement system based on a mobile acquisition device includes: a mobile acquisition device, a data preprocessing module, a target detection module, a point cloud stitching module, a ground point segmentation module, a target point cloud segmentation module, a vehicle contour point segmentation module, a first size calculation module, and a second size calculation module.

[0166] The mobile data acquisition device is equipped with a lidar and a camera for data collection.

[0167] The data preprocessing module is used to perform time synchronization and joint calibration of the lidar and camera on the mobile acquisition device;

[0168] The target detection module is used to train the target detection model and then use the trained target detection model to detect targets in the image data collected by the camera, obtaining the target vehicle position as the target detection result.

[0169] The point cloud stitching module is used to control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform each frame of point cloud in the movement process to the coordinate system of the first frame according to the rotation transformation matrix. The point cloud data of each frame is stitched together to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline.

[0170] The ground point segmentation module is used to segment the ground point cloud into non-ground point clouds;

[0171] The target point cloud segmentation module is used to segment the point cloud of the target vehicle in the non-ground point cloud by combining the target detection results obtained by the target detection module, so as to obtain the target point cloud.

[0172] The vehicle contour point segmentation module is used to extract vehicle contour points from the target point cloud.

[0173] The first dimension calculation module is used to fit the length L and width W of the target vehicle's outer dimensions based on the yaw angle of the vehicle's outline points, and to obtain the center point of the vehicle's outline on the plane xoy.

[0174] The second dimension calculation module is used to traverse the target point cloud obtained by the target point cloud segmentation module to obtain its maximum value Zmax on the Z-axis. It combines the ground point cloud segmented by the ground point segmentation module and the center point of the car outline on the plane xoy obtained by the first dimension calculation module to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, the height H in the outer dimensions of the target vehicle is obtained.

[0175] It should be noted that:

[0176] In this embodiment, a LiDAR and a camera are installed on the mobile acquisition device. The size algorithm for LiDAR point cloud stitching and visual fusion is deployed in an NVIDIA embedded platform. The mobile acquisition device, as shown in the example... Figure 2 As shown, the power bank powers the OS1-64-line LiDAR, the InterRealSense D435i camera, the Nvidia Jetson AGX Xavier embedded platform, and the display. The OS1-64-line LiDAR has a horizontal field of view of 360°, a vertical field of view of ±22.5°, a horizontal angular resolution of 0.35°, a vertical angular resolution of 0.7°, a ranging accuracy of ±1cm within 1–20 meters, and a scanning frequency of 10Hz. The InterRealSense camera has a field of view of 69°×42°, a resolution of 1280×720, and a frame rate of 30 frames / s.

[0177] In addition, in this embodiment, the target detection module uses the YOLOv7-tiny network as the target detection model, such as... Figure 5 As shown, the specific tasks of the object detection module in training, testing, and deploying the object detection model include:

[0178] (1) Dataset Construction: Images of cars were acquired through web scraping, camera photography, and other methods. The images were cleaned to remove low-quality and duplicate images. The X-Anylabeling tool was used to label the images, resulting in a target detection dataset with 3684 images and 8580 labels. The dataset was then divided into training and validation sets in an 8:2 ratio.

[0179] (2) Model Establishment: The YOLOv7-tiny network mainly consists of four parts: input, backbone, neck, and head. First, after data augmentation and other preprocessing operations, the input part performs necessary preprocessing operations on the original image. Next, the preprocessed image is sent to the backbone for feature extraction, which is mainly constructed by convolution, E-ELAN module, MPConv module, and SPPCSPC module. After extracting feature maps of different scales from the backbone, these features are effectively fused through the neck module to obtain feature maps of large, medium, and small sizes. The neck module adopts the traditional PA-FPN structure. The feature maps fused into the neck are sent to the head of three different sizes to output detection boxes, and NMS is used to remove redundant boxes to obtain the detection results. The detection head introduces RepConv, which introduces the concept of reparameterization. During training, it learns in multiple branches, and during inference, the parameters are merged into one branch to speed up the inference speed.

[0180] (3) Model training, testing and deployment: First, build a PyTorch environment on a server with an NVIDIA GeForce RTX 3090 24GB graphics card, configure the environment required by the YOLOv7 repository, create a dataset parameter yaml file, set the input image size to 640*640, batch_size to 16, the number of training epochs to 200, and set the other hyperparameters to the official default settings. Wait for the model to finish training and observe the model loss curve. If it does not converge, continue training. Select the weights with the lowest training loss to test the model. Set the threshold for IOU to 0.45 and the threshold for confidence to 0.5. Output the inference results of the model on the validation set and output the model's mAP, precision and recall through the precision evaluation program. Before deployment, reparameterize the model and export the weight parameters of each layer of the model. Build the YOLOv7-tiny inference model based on the TensorRT framework and deploy it to the embedded platform. Encapsulate it as a ROS node to facilitate the receiving of camera data.

[0181] In (1), various networks were selected on the dataset, and the results are shown in Table 1. It can be seen that the YOLOv7-tiny model has higher performance than other models, and its speed can reach 55FPS in the embedded platform, which can achieve real-time high-precision detection effect.

[0182] Table 1 Target Detection Model Selection

[0183]

[0184] The results of applying this invention to automobile dimension measurement are shown in Table 2, with specific parameter selections shown in Table 3. The resulting graphs are shown in Figures 6-7. Figure 6 This is a schematic diagram of the single-target measurement results. Figure 7 The diagram shows the results of multi-target measurement. Experimental results show that the measurement accuracy of the present invention meets the requirements of GB38900-2020 for automatic measuring devices to measure the size of automobiles within ±2% or ±100mm. Excluding the time required for moving the trolley, the size measurement algorithm runs for less than 0.5s, realizing a technology for measuring the external dimensions of automobiles with high measurement accuracy, fast running speed and wide measurement range.

[0185] Table 2. Vehicle Dimension Measurement Results

[0186]

[0187] Table 3 Parameter Selection

[0188]

[0189]

[0190] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for measuring vehicle dimensions based on a mobile data acquisition device, wherein the mobile data acquisition device is equipped with a camera and a lidar for acquiring data, characterized in that, Includes the following steps: S1. The lidar and camera undergo time synchronization and joint calibration; S2. The trained target detection model is used to detect targets in the image data collected by the camera, and the position of the target vehicle is obtained as the target detection result. S3. Control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform the point cloud of each frame during the movement to the coordinate system of the first frame according to the rotation transformation matrix. Then, stitch together the point cloud data of each frame to obtain a global map to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline. S4. Segment the global map to obtain ground point cloud and non-ground point cloud; S5. Combining the target detection results obtained in S2, segment the point cloud of the target vehicle in the non-ground point cloud to obtain the target point cloud; S6. Extract the vehicle contour points from the target point cloud; S7. Fit the length L and width W of the target vehicle's outline dimensions based on the yaw angle of the vehicle's outline points, and obtain the center point of the vehicle outline on the xoy plane. S8. Traverse the target point cloud obtained in S5 to obtain its maximum value Zmax on the Z-axis. Combine the ground point cloud segmented in S4 and the center point of the car contour on the xoy plane obtained in S7 to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, obtain the height H in the outer dimensions of the target vehicle.

2. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S1 includes: Data collected by the camera and LiDAR nodes on the mobile acquisition device is synchronized in time. Furthermore, by acquiring checkerboard-patterned images and point cloud data from different locations, the intrinsic parameters Pr of the camera and the extrinsic parameters Tr of the LiDAR and camera are calibrated. Pr and Tr are: where f x and f y are the focal lengths of the camera coordinate system x and y axes, u0 and v0 are the actual positions of the principal point on the x and y axes, R is the rotation matrix of the laser-to-camera, and T is the translation matrix of the laser-to-camera.

3. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S3 includes: S31. Transfer the point cloud from the previous frame S c ={a0,...,a N } and the point cloud T of the next frame c ={b0,...,b N The surface containing the point is modeled as a Gaussian distribution, such that it satisfies... and S32. Initialize the rotation transformation matrix, set the point cloud resolution, and transform the point cloud T... c Voxelization is performed, and the mean μ, variance C, and number of points N in each voxel are recorded. S33. Regarding S c The point cloud is traversed, and the voxel position index of the current point is calculated based on the point cloud resolution. If it is in T c If the value is not found in the function, skip the current point and calculate the error E based on the optimization function F. i and Hessian matrix H i The process is then accumulated, and after traversal, a Newton-Gaussian iteration is performed. If the rotation transformation matrix does not converge, step S33 is repeated. The optimization function is: In the formula, T is the translation matrix from the lidar to the camera, and N... i For a i Nearby T c The number of point clouds in a voxel, J i Let Ta be a Jacobian matrix. i |x l Point a i Convert to point cloud T c The value of the x-axis in the coordinate system; Ta i |y l Point a i Convert to point cloud T c The value of the y-axis in the coordinate system; S34. Transform the point cloud at each time step to the coordinate system of the first frame, where the transformation matrix from each frame to the first frame is Rt. n =(RT) t0 ×RT t1 ×....×RT tn ) -1 Once the mobile data acquisition device stops moving, the global map stitching is complete, and the car outlines are reconstructed simultaneously.

4. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S4 includes: S41. Project the unordered point cloud data onto the xoy plane to transform it into an ordered data representation. The xoy plane is considered as a circle with an infinite radius. Divide the circle (with radii greater than r_min and less than r_max) into n_segments of sectors. Within each sector, divide it into n_bins of rings based on distance. In each ring, transform the point cloud {x,y,z} into {d,z} to achieve dimensionality reduction. The dimensionality reduction process is as follows: S42. Traverse each sector, starting from the first sector ring to fit a straight line. Use the maximum slope of the line (max_slope), the fitting error of the line (max_error_square), and the intercept of the line on the z-axis to select points to add to the line. Use the longest distance threshold of the line (long_threshold) to determine whether to start fitting a new line, until the line fitting in each sector is completed. S43. Traverse the point cloud. If the distance from a point to a line is less than the set maximum distance max_dist_to_line, it is considered a ground point. Based on the correspondence between the reduced point cloud and the original point cloud, the ground points and non-ground points in the original point cloud are deduced. S44. Calculate the centroid {x} of the ground point. g ,y g ,z g }, set ground_threshhold, and further filter out z-values ​​from non-ground point clouds using a pass-through filter. g Point cloud below the +ground_threshhold height.

5. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S5 includes: S51. Transform the point cloud from the lidar coordinate system to the pixel coordinate system. If the point cloud falls within the two-dimensional bounding box of the target detection result obtained in S2, it is determined to be a point to be clustered. Each two-dimensional detection box in the image has a corresponding cluster of point clouds to be clustered. S52. Cluster each point cloud cluster to be clustered to obtain several point cloud clusters. Project the centroid of the point cloud cluster onto the image and select the point cloud cluster with the smallest distance between the centroid and the center of the two-dimensional detection box as the target point cloud. S53. Construct the target point cloud into an ocTree tree with a resolution of tree_resolution. Traverse the target point cloud and perform a spherical neighborhood search with radius radius for each point in the ocTree tree. If the number of neighborhoods is less than erode_count, it is judged as a noise point. Finally, remove all noise points from the target point cloud.

6. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 5, characterized in that, The specific steps involved in transforming the point cloud from the lidar coordinate system to the pixel coordinate system in S51 include: Let the coordinates of any point P in the lidar coordinate system be (P x ,P y ,P z ), whose coordinates in the camera coordinate system are (X w ,Y w Z w Given a point P with coordinates (x, y) in the pixel coordinate system, the transformation relationship from the lidar coordinate system to the pixel coordinate system is as follows: Where Pr is the intrinsic parameter of the camera, and Tr is the extrinsic parameter of the lidar to the camera.

7. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S6 includes: Project the target point cloud onto the xoy plane and use the ConvexHull convex hull algorithm to find the contour points of the point cloud. Traverse all contour points, build an Octree tree to find the point cloud within a spherical region of radius radius, calculate the normal vector, and if the angle between the normal vector of the point cloud and the normal vector of the contour exceeds contour_deg, then the point is determined to be a convex part. All points identified as protruding parts are filtered out from the target point cloud. After erosion and denoising of the remaining target point cloud, the ConvexHull convex hull algorithm is used again to find the contour points of the point cloud.

8. The method for measuring automobile dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S7 includes: Traverse the range 0 to π / 2 with a step size of reasearch_deg, rotate the contour point cloud by an angle of -θ, and obtain the values ​​in two dimensions, C1 and C2: In the formula, θ = n × research_deg, and Q is the set of contour points {(x i ,y i ,z i )|i=0,1,2,…,m}, where m is the number of contour points; Based on the minimum area criterion, the loss value is recorded for each step length. The loss function is: Loss=(max(C1)-min(C1))×(max(C2)-min(C2)) (12) After the traversal is completed, the yaw angle θ corresponding to the minimum loss is selected. Then the corresponding length L is max(C1)-min(C1), the width W is max(C2)-min(C2), the center point cx on the X-axis is (max(C1)+min(C1)) / 2, and the center point cy on the Y-axis is (max(C2)+min(C2)) / 2.

9. The method for measuring vehicle dimensions based on a mobile acquisition device according to claim 1, characterized in that, The specific content of S8 includes: The target point cloud obtained by S5 is traversed to obtain its maximum value Zmax on the Z-axis; A KD-tree is constructed from the ground points obtained in S4. Based on the center point on the xoy plane obtained in S7, the nearest neighbor search algorithm is used to find n_neighbors of its nearest point cloud. The ground plane around the car is fitted based on the found point cloud, where the ground plane direction is: z = ax + by + c (13) In the formula, a, b, and c are the coefficients of the plane. According to the principle of least squares fitting, the sum of the residuals of the plane fitting is: When E takes its minimum value but: Solving the above equation yields the plane equation coefficients a, b, and c, from which the height H of the car can be calculated: H=Zmax-(a×cx+b×cy+c) (16).

10. A vehicle dimension measurement system based on a mobile acquisition device, characterized in that, include: Mobile acquisition device, data preprocessing module, target detection module, point cloud stitching module, ground point segmentation module, target point cloud segmentation module, vehicle outline point segmentation module, first dimension calculation module, and second dimension calculation module; The mobile data acquisition device is equipped with a lidar and a camera for data collection. The data preprocessing module is used to perform time synchronization and joint calibration of the lidar and camera on the mobile acquisition device; The target detection module is used to train the target detection model and then use the trained target detection model to detect targets in the image data collected by the camera, obtaining the target vehicle position as the target detection result. The point cloud stitching module is used to control the mobile acquisition device to move around the target vehicle at least half a circle, obtain the rotation transformation matrix between two adjacent frames, and then transform each frame of point cloud in the movement process to the coordinate system of the first frame according to the rotation transformation matrix. The point cloud data of each frame is stitched together to complete the three-dimensional reconstruction of the current space and reproduce the point cloud information of the car outline. The ground point segmentation module is used to segment the ground point cloud into non-ground point clouds; The target point cloud segmentation module is used to segment the point cloud of the target vehicle in the non-ground point cloud by combining the target detection results obtained by the target detection module, so as to obtain the target point cloud. The vehicle contour point segmentation module is used to extract vehicle contour points from the target point cloud. The first dimension calculation module is used to fit the length L and width W of the target vehicle's outer dimensions based on the yaw angle of the vehicle's outline points, and to obtain the center point of the vehicle's outline on the xoy plane. The second dimension calculation module is used to traverse the target point cloud obtained by the target point cloud segmentation module to obtain its maximum value Zmax on the Z-axis. It combines the ground point cloud segmented by the ground point segmentation module and the center point of the car outline on the plane xoy obtained by the first dimension calculation module to obtain the plane equation of the ground plane around the car. Based on the difference between Zmax and the plane equation, the height H in the outer dimensions of the target vehicle is obtained.

Images