Accident vehicle overall dimension measurement method based on depth camera

Abstract

The invention relates to an accident vehicle overall dimension measurement method based on a depth camera. The accident vehicle overall dimension measurement method comprises the following steps: S1, continuously collecting multi-angle outer contour images of an accident vehicle through the depth camera; s2, preprocessing the external contour image of each angle to obtain a color image and a first depth image of each angle of the accident vehicle; s3, segmenting the color image of each angle, and mapping the segmented color image to the first depth image of the same angle to generate a second depth image; s4, converting the second depth images of the multiple angles into three-dimensional point cloud data, and registering and splicing the three-dimensional point cloud data to obtain outer contour three-dimensional point cloud data of the accident vehicle; s5, clustering and screening the three-dimensional point cloud data, and extracting the outer contour of the accident vehicle through a boundary extraction method; s6, the length of the accident vehicle in the direction of each three-dimensional coordinate axis is obtained, and then the overall dimension of the accident vehicle is obtained. The overall dimension of the vehicle can be accurately measured.

Description

Technical Field

[0001] The present invention relates to the field of image information processing, and in particular to a method for measuring the outer dimensions of an accident vehicle based on a depth camera. Background Art

[0002] In the process of handling traffic accidents and vehicle insurance claims, accurately measuring the external dimensions of the accident vehicle is a crucial step. This dimensional information is of great significance for determining accident responsibility, assessing vehicle losses, and guiding repair work.

[0003] The commonly used measurement methods at the accident scene mainly include contact and non-contact methods. Traditional vehicle size measurement methods have many shortcomings. Contact measurement methods, such as manual measurement with a tape measure, are not only time-consuming and labor-intensive, but also require the coordinated operation of multiple staff members. In addition, the measurement results are easily affected by human factors, such as operating errors or visual errors, resulting in insufficient measurement accuracy. In addition, at the accident scene, this method may also increase the safety risks of the staff because they need to operate at close range in a potentially dangerous environment. Non-contact measurement methods, such as laser rangefinder measurement, although they can provide higher measurement accuracy, have higher equipment costs and can only perform single-point measurement, which is less efficient. In addition, this method requires high professional skills of the operator and requires special training to be proficient in use. The traditional camera measurement method is easily affected by ambient lighting conditions, and due to the lack of depth information, the measurement accuracy is limited. At the same time, this method requires a complex camera calibration process, and the operation is cumbersome. Summary of the invention

[0004] The purpose of the present invention is to provide a method for measuring the outer dimensions of an accident vehicle based on a depth camera, so as to solve the problem that the traditional camera measurement method is easily affected by external environmental factors, resulting in inaccurate measurement results.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a method for measuring the outer dimensions of an accident vehicle based on a depth camera, comprising the following steps:

[0006] S1: Continuously collect the outline images of the accident vehicle from multiple angles through the depth camera;

[0007] S2: Preprocessing the outline images at various angles respectively to obtain color images and first depth images of the accident vehicle at various angles;

[0008] S3: performing segmentation processing on the color image at each angle, and mapping the segmented color image to the first depth image at the same angle to generate a second depth image;

[0009] S4: converting the second depth images at multiple angles into three-dimensional point cloud data, and registering and splicing the three-dimensional point cloud data to obtain three-dimensional point cloud data of the outer contour of the accident vehicle;

[0010] S5: clustering and screening the three-dimensional point cloud data of the outer contour of the accident vehicle, and extracting the outer contour of the accident vehicle by using a boundary extraction method;

[0011] S6: Establishing the stereo coordinate axes of the preprocessed three-dimensional point cloud data based on the edge contour of the accident vehicle, obtaining the length of the accident vehicle along the direction of each stereo coordinate axis, and then obtaining the outer dimensions of the accident vehicle.

[0012] Through the above technical means, the present invention introduces depth information so that the measurement method can more accurately reflect the three-dimensional shape and size of the accident vehicle, overcoming the problem that the traditional camera measurement method is easily affected by external environmental factors and leads to inaccurate measurement results. At the same time, the present invention uses the color images and depth information obtained from multiple angles by the RGB-D camera, and through the registration and splicing of three-dimensional point cloud data, it can reflect the fine structure and contour features of the surface of the accident vehicle, and achieve high-precision three-dimensional reconstruction.

[0013] The present invention preprocesses the collected images to improve image quality, and improves the accuracy of depth data by generating a second depth image. It uses automated image processing and three-dimensional reconstruction technology to reduce the need for manual measurement and errors caused by human operation, reduce the safety risks of on-site workers, and improve the efficiency and accuracy of measurement.

[0014] Furthermore, the S1 specifically includes:

[0015] S11: using a depth camera to perform surround capture of the accident vehicle according to a preset interval angle, and obtaining contour images of the accident vehicle at multiple angles;

[0016] S12: Perform integrity check and overlapping area check on the collected contour image. If the overlapping area is not less than the set value, the total coverage angle is 360°, and the measured angle difference is less than the preset angle, then enter step S2; otherwise, re-collect the image.

[0017] Through the above-mentioned technical means, by detecting whether the total coverage angle of the collected contour image reaches 360°, it is ensured that all angles of the vehicle are covered, avoiding inaccurate measurements caused by missing images; by detecting whether the overlapping area between adjacent images meets the set value, it is ensured that in the subsequent image registration and stitching process, adjacent images have sufficient overlapping parts, thereby improving the accuracy of registration and the continuity of stitching; by calculating the measurement angle difference between adjacent images and ensuring that it is less than the preset angle, the density and uniformity of image acquisition are controlled, which helps to improve the accuracy and robustness of three-dimensional reconstruction.

[0018] Furthermore, the integrity detection includes the following steps: sorting the contour images according to the acquisition angle to obtain the total coverage angle of all the contour images; and calculating the angle difference between two contour images with adjacent angles.

[0019] Through the above-mentioned technical means, by checking the total coverage angle and measurement angle difference of the image, it is ensured that the collected data can completely cover the entire vehicle, which helps to avoid errors caused by inaccurate image acquisition and improves the robustness of the entire measurement process; by ensuring uniform coverage of the image and accurate acquisition angle difference, the reliability of the entire measurement process is improved, and problems caused by improper acquisition are reduced. The automated acquisition control reduces the need for human judgment and operation, making the entire measurement process simpler and more intuitive; at the same time, data integrity and accuracy are checked during the image acquisition stage, which reduces the complexity and amount of calculation in data processing and improves the overall data processing efficiency.

[0020] Furthermore, the overlapping area detection comprises the following steps:

[0021] Extract feature points from each of the outline images, perform feature matching on two angularly adjacent outline images, remove mismatched feature points, and generate a plurality of homography matrices between two angularly adjacent outline images; obtain the intersection area of ​​the bounding boxes of two angularly adjacent outline images according to each of the homography matrices, and calculate the ratio of the intersection area of ​​the bounding boxes to the total area of ​​the outline image.

[0022] Through the above-mentioned technical means, by extracting feature points and matching features, the accuracy of matching between images is enhanced, and by removing erroneous feature points, the reliability of the feature matching process is ensured, and the errors caused by erroneous matching are reduced; through precise feature point matching and calculation of the homography matrix, the method can adapt to different vehicle shapes and accident scene conditions, thereby improving the adaptability and flexibility of the method.

[0023] Furthermore, the outline image at each angle includes an original depth image and an original color image.

[0024] Through the above technical means, the original depth image provides the depth information of the scene, while the original color image provides the color and texture information of the scene. The combination of the original depth image and the original color image can provide richer visual information, which helps to improve the accuracy and reliability of the outline image.

[0025] Furthermore, the S2 specifically includes the following steps:

[0026] Performing noise reduction processing on the original depth image and data cleaning, filtering out data points with a depth value of 0 or invalid in the original depth image, so as to obtain the first depth image of each angle of the accident vehicle;

[0027] The original color image is enhanced and distortion corrected to obtain the color image of the accident vehicle at various angles.

[0028] Through the above-mentioned technical means, by performing denoising and data cleaning on each of the original depth images, it is helpful to remove noise and abnormal data in the image and improve the quality of the first depth image. Traversing data cleaning helps to remove outliers and irrelevant information, ensuring that the data of the first depth image is more accurate and reliable; performing image enhancement and distortion correction on each of the original color images can speed up image processing, thereby improving the efficiency of the overall measurement process.

[0029] Furthermore, the S3 specifically includes the following steps:

[0030] S31: performing panoramic segmentation on each of the color images to segment out the accident vehicle part;

[0031] S32: Aligning each of the segmented color images with each of the first depth images, and performing mask processing on the accident vehicle segmented in each of the color images to obtain a depth value of a corresponding position of the accident vehicle, and thereby obtaining the corresponding second depth images.

[0032] Through the above-mentioned technical means, panoramic segmentation is performed through a pre-trained neural network model, which can accurately segment the accident vehicle from the color image, and align the segmented color image with the first depth image to ensure the spatial consistency of the color information and depth information; through precise image segmentation and depth information fusion, the data processing process is optimized, thereby improving the efficiency of the overall measurement process.

[0033] Furthermore, the S4 specifically includes the following steps:

[0034] S41: Converting each of the second depth images into each of the three-dimensional point cloud data according to the built-in parameters of the depth camera;

[0035] S42: extracting a plurality of feature points from each of the three-dimensional point cloud data;

[0036] S43: selecting a plurality of matching points from each of the feature points, calculating a rigid transformation between the matching points, and obtaining a transformation matrix through multiple iterations;

[0037] S44: Optimizing the transformation matrix until the registration error between the matching points is lower than a set threshold or reaches a preset number of iterations, and obtaining a final change matrix;

[0038] S45: splicing the three-dimensional point cloud data according to the final change matrix to obtain the three-dimensional point cloud data of the accident vehicle;

[0039] S45: removing outliers from the three-dimensional point cloud data of the accident vehicle to obtain the outer contour three-dimensional point cloud data, and then fitting a geometric surface model of the vehicle body according to the outer contour three-dimensional point cloud data.

[0040] Through the above technical means, the second depth image is converted into three-dimensional point cloud data through the built-in parameters of the depth camera, ensuring the consistency of the spatial coordinates of the data with the actual scene; the three-dimensional point cloud data are matched using the progressive consistent sampling algorithm and the iterative closest point algorithm, which improves the efficiency and accuracy of data processing and ensures the consistency and accuracy of the point cloud data; the first matching is performed through the progressive consistent sampling algorithm, and then the result is matched for the second time according to the iterative closest point algorithm, which enhances the registration accuracy of the point cloud data under different viewing angles; the point cloud data from multiple angles are spliced ​​into a complete three-dimensional point cloud data of the outer contour of the accident vehicle, providing comprehensive geometric information of the accident vehicle.

[0041] Furthermore, the S5 specifically includes the following steps:

[0042] S51: clustering the outer contour three-dimensional point cloud data to extract three-dimensional point cloud data describing the body of the accident vehicle;

[0043] S52: Extracting the outline of the accident vehicle from the three-dimensional point cloud data describing the body of the accident vehicle by using a boundary extraction method.

[0044] Through the above-mentioned technical means, the three-dimensional point cloud data is clustered, and the vehicle-related point cloud data is effectively separated from the background or irrelevant data, thereby improving the accuracy of data processing; the cluster containing the vehicle body is screened out from the clustering results, so that the point cloud data of the accident vehicle can be accurately located; the outline of the accident vehicle is extracted from the screened vehicle body clusters through the boundary extraction method. This step ensures the accurate recognition of the vehicle's outer contour.

[0045] Furthermore, the S6 specifically includes the following steps:

[0046] S61: Convert the three-dimensional point cloud data describing the body of the accident vehicle into a matrix representation to obtain a three-dimensional point cloud data matrix, wherein each row in the three-dimensional point cloud data matrix represents the coordinates of each point;

[0047] S62: decomposing the three-dimensional point cloud data matrix into a low-rank matrix and a sparse matrix, obtaining the main directions of the three-dimensional point cloud data by singular value decomposition of the low-rank matrix, and then establishing a three-dimensional coordinate system and obtaining the three main axis directions of the three-dimensional coordinate system;

[0048] S63: construct multiple directed bounding boxes in each of the main axis directions based on the edge contour of the accident vehicle and the minimum volume directed bounding box algorithm, obtain the total length of the multiple directed bounding boxes in each of the main axis directions, and then obtain the outer dimensions of the accident vehicle.

[0049] Through the above-mentioned technical means, the three-dimensional point cloud data matrix is ​​decomposed into a low-rank matrix and a sparse matrix using singular value decomposition technology, and then the main direction of the three-dimensional data is effectively extracted through singular value decomposition of the low-rank matrix, thereby improving the efficiency of data processing; by constructing multiple directed bounding boxes in each main axis direction and using the minimum volume directed bounding box algorithm, the box that is closest to the vehicle contour can be found, thereby obtaining the smallest volume, thereby improving the efficiency and accuracy of volume calculation; directly constructing a directed bounding box on the three-dimensional point cloud data simplifies the data processing process and improves the speed and efficiency of data processing.

[0050] Beneficial effects of the present invention:

[0051] 1. The present invention introduces depth information so that the measurement method can more accurately reflect the three-dimensional shape and size of the accident vehicle, overcoming the problem that the traditional camera measurement method is easily affected by external environmental factors, resulting in inaccurate measurement results. At the same time, the present invention uses the color images and depth information obtained from multiple angles by the RGB-D camera, and through the registration and splicing of three-dimensional point cloud data, it can reflect the fine structure and contour features of the surface of the accident vehicle, and achieve high-precision three-dimensional reconstruction.

[0052] 2. The present invention pre-processes the collected images to improve the image quality, and improves the accuracy of the depth data by generating a second depth image. It uses automated image processing and three-dimensional reconstruction technology to reduce the need for manual measurement and errors caused by human operation, reduce the safety risks of on-site workers, and improve the efficiency and accuracy of the measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of a specific implementation process of the present invention;

[0054] Figure 2 It is a schematic diagram of the implementation process of image acquisition in the present invention;

[0055] Figure 3 It is a schematic diagram of the preprocessing process of each image in the present invention;

[0056] Figure 4 A schematic diagram of the process of obtaining the three-dimensional point cloud data of the outer contour of the accident vehicle in the present invention;

[0057] Figure 5 A schematic diagram of a process for obtaining the outline of an accident vehicle in the present invention;

[0058] Figure 6 A schematic diagram of a process for obtaining the outer dimensions of an accident vehicle in the present invention;

[0059] Figure 7 It is a specific schematic diagram when collecting images in this embodiment. DETAILED DESCRIPTION

[0060] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0061] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0062] like Figure 1 As shown, in this embodiment, a method for measuring the outer dimensions of an accident vehicle based on a depth camera includes the following steps:

[0063] S1: Continuously collect the outline images of the accident vehicle from multiple angles through the depth camera;

[0064] S2: Preprocessing the outline images at various angles respectively to obtain color images and first depth images of the accident vehicle at various angles;

[0065] S3: performing segmentation processing on the color image at each angle, and mapping the segmented color image to the first depth image at the same angle to generate a second depth image;

[0066] S4: converting the second depth images at multiple angles into three-dimensional point cloud data, and registering and splicing the three-dimensional point cloud data to obtain three-dimensional point cloud data of the outer contour of the accident vehicle;

[0067] S5: clustering and screening the three-dimensional point cloud data of the outer contour of the accident vehicle, and extracting the outer contour of the accident vehicle by using a boundary extraction method;

[0068] S6: Establishing the stereo coordinate axes of the preprocessed three-dimensional point cloud data based on the edge contour of the accident vehicle, obtaining the length of the accident vehicle along the direction of each stereo coordinate axis, and then obtaining the outer dimensions of the accident vehicle.

[0069] By introducing depth information, the measurement method can more accurately reflect the three-dimensional shape and size of the accident vehicle, overcoming the problem that the traditional camera measurement method is easily affected by external environmental factors, resulting in inaccurate measurement results. At the same time, the present invention uses the color images and depth information obtained from multiple angles by the RGB-D camera, and through the registration and splicing of three-dimensional point cloud data, it can reflect the fine structure and contour features of the surface of the accident vehicle, and achieve high-precision three-dimensional reconstruction.

[0070] The captured images are preprocessed to improve image quality, and the accuracy of depth data is improved by generating a second depth image. The use of automated image processing and 3D reconstruction technology reduces the need for manual measurement and errors caused by human operation, reduces safety risks for on-site workers, and improves measurement efficiency and accuracy.

[0071] Preferably, the depth camera is an RGB-D camera.

[0072] like Figure 2 As shown, in this embodiment, S1 specifically includes:

[0073] S11: using a depth camera to perform surround capture of the accident vehicle according to a preset interval angle, and obtaining contour images of the accident vehicle at multiple angles;

[0074] S12: Perform integrity check and overlapping area check on the collected contour image. If the overlapping area is not less than the set value, the total coverage angle is 360°, and the measured angle difference is less than the preset angle, then enter step S2; otherwise, re-collect the image.

[0075] By detecting whether the total coverage angle of the collected contour images reaches 360°, it is ensured that all angles of the vehicle are covered, avoiding inaccurate measurements due to missing images; by detecting whether the overlapping area between adjacent images meets the set value, it is ensured that in the subsequent image registration and stitching process, adjacent images have sufficient overlapping parts, thereby improving the accuracy of registration and the consistency of stitching; by calculating the measurement angle difference between adjacent images and ensuring that it is less than the preset angle, the density and uniformity of image acquisition are controlled, which helps to improve the accuracy and robustness of 3D reconstruction.

[0076] like Figure 7 As shown, preferably, the image acquisition process specifically includes: setting up the RGB-D camera at a distance of 2-3 meters from the vehicle, with the camera lens facing the vehicle; during the image acquisition process, the operator needs to perform a surround acquisition around the accident vehicle, and collect a set of data on the outer contour of the accident vehicle with a measurement angle difference less than a preset angle difference, ensuring that the overlapping area between adjacent views is not less than a set value for subsequent alignment, and collecting 3-5 frames of data from each viewing angle for screening the best image.

[0077] Preferably, the preset angle difference is 45°, that is, it is necessary to collect images of the accident vehicle's outline from at least 8 angles, and the overlapping area setting value is 30%.

[0078] In other embodiments, the preset angle difference of the adjacent contour images at each angle can be adjusted according to the actual object to be measured and the actual scene, and the overlapping area setting value between the adjacent contour images at each angle can be adjusted to better capture the image of the object to be measured, thereby better measuring the contour size of the object to be measured.

[0079] In this embodiment, the integrity detection includes the following steps: sorting the contour images according to the acquisition angle to obtain the total coverage angle of all the contour images; and calculating the angle difference between two contour images with adjacent angles.

[0080] By checking the total coverage angle and measurement angle difference of the image, it is ensured that the collected data can fully cover the entire vehicle, which helps to avoid errors caused by inaccurate image acquisition and improves the robustness of the entire measurement process; by ensuring uniform coverage of the image and accurate acquisition angle difference, the reliability of the entire measurement process is improved, and problems caused by improper acquisition are reduced. The automated acquisition control reduces the need for human judgment and operation, making the entire measurement process simpler and more intuitive; at the same time, data integrity and accuracy checks are performed during the image acquisition stage, which reduces the complexity and computational complexity of data processing and improves overall data processing efficiency.

[0081] Preferably, one or more algorithms of a fast scale-space-based feature extraction algorithm (AKAZE algorithm), a binary robust invariant scalable key point (BRISK algorithm) and a superpoint algorithm (SuperPoint algorithm) are selected according to specific circumstances to extract feature points from each image. The AKAZE algorithm can improve the computational efficiency while maintaining the stability and repeatability of feature points by combining the nonlinear diffusion characteristics and acceleration strategy of a scale-space-based feature extraction algorithm (KAZE algorithm); the BRISK algorithm is based on binary descriptors, has high computational efficiency, compact descriptors, and is robust to changes in illumination and perspective; the SuperPoint algorithm is a feature point detection and description algorithm based on deep learning, which extracts feature points and descriptors through a convolutional neural network (CNN) and has high robustness and accuracy for complex scenes. These algorithms can provide better performance and more stable feature matching results when processing large-scale data and complex scenes.

[0082] In this embodiment, the overlapping area detection includes the following steps:

[0083] Extract feature points from each of the outline images, perform feature matching on two angularly adjacent outline images, remove mismatched feature points, and generate a plurality of homography matrices between two angularly adjacent outline images; obtain the intersection area of ​​the bounding boxes of two angularly adjacent outline images according to each of the homography matrices, and calculate the ratio of the intersection area of ​​the bounding boxes to the total area of ​​the outline image.

[0084] By extracting feature points and matching features, the accuracy of matching between images is enhanced, and by removing erroneous feature points, the reliability of the feature matching process is ensured and the errors caused by erroneous matching are reduced; through precise feature point matching and calculation of the homography matrix, the method can adapt to different vehicle shapes and accident scene conditions, thereby improving the adaptability and flexibility of the method.

[0085] In this embodiment, the outline image at each angle includes an original depth image and an original color image.

[0086] The original depth image provides depth information of the scene, while the original color image provides color and texture information of the scene. The original depth image and the original color image are used in combination to provide richer visual information, which helps to improve the accuracy and reliability of the outline image.

[0087] In this embodiment, S2 specifically includes the following steps:

[0088] Performing noise reduction processing on the original depth image and data cleaning, filtering out data points with a depth value of 0 or invalid in the original depth image, so as to obtain the first depth image of each angle of the accident vehicle;

[0089] The original color image is enhanced and distortion corrected to obtain the color image of the accident vehicle at various angles.

[0090] By performing denoising and data cleaning on each of the original depth images, it is helpful to remove noise and abnormal data in the image and improve the quality of the first depth image. Traversing data cleaning helps to remove outliers and irrelevant information, ensuring that the data of the first depth image is more accurate and reliable; performing image enhancement and distortion correction on each of the original color images can speed up image processing, thereby improving the efficiency of the overall measurement process.

[0091] Preferably, guided filtering is used to reduce noise in the original depth image. Guided filtering is a filtering method based on a local linear model. By establishing a local linear model in the neighborhood of each pixel, it can effectively remove noise while maintaining edge information. Compared with traditional bilateral filtering, guided filtering has lower computational complexity and faster running speed.

[0092] More preferably, the filter window size is set to 5×5, the spatial domain standard deviation is 3, and the value range standard deviation is 0.1.

[0093] Preferably, each pixel in the depth image is traversed, its depth value is checked, and the contour image is cleaned to obtain a first depth image by removing the depth values ​​of each contour image that are 0 or exceed the valid range (for example, exceed the maximum measurement distance of the sensor or are less than the minimum measurement distance), thereby ensuring that the data of the first depth image is more accurate and reliable.

[0094] Preferably, a scale image enhancement algorithm is used to enhance the original color image. By processing the image at multiple scales, the details and structure of the image can be better preserved while enhancing the contrast and brightness.

[0095] Preferably, the image correction specifically includes the following steps: obtaining the camera's intrinsic parameter matrix and distortion coefficient through a calibration process, wherein the intrinsic parameter matrix includes parameters such as focal length and principal point coordinates, and the distortion coefficient describes the radial distortion and tangential distortion of the lens; then, according to the camera's intrinsic parameter matrix and distortion coefficient, calculating the coordinates of each pixel point under ideal distortion-free conditions; then, using the reverse mapping method, mapping the corrected image coordinates back to the distortion coordinates in the original image, that is, using a polynomial model to describe radial distortion and tangential distortion, and obtaining the corrected pixel coordinates by solving the polynomial equation; finally, generating a distortion-free image by interpolating the corrected coordinates. Through this process, the influence of lens distortion on the image can be effectively eliminated, the straight lines in the image can be restored to straight lines, the geometric accuracy and visual effect of the image can be improved, and the distortion correction not only helps the subsequent image processing and analysis, but also improves the accuracy of tasks such as feature point detection, image registration, and three-dimensional reconstruction.

[0096] like Figure 3 As shown, in this embodiment, S3 specifically includes the following steps:

[0097] S31: performing panoramic segmentation on each of the color images to segment out the accident vehicle part;

[0098] S32: Aligning each of the segmented color images with each of the first depth images, and performing mask processing on the accident vehicle segmented in each of the color images to obtain a depth value of a corresponding position of the accident vehicle, and thereby obtaining the corresponding second depth images.

[0099] By using a pre-trained neural network model for panoramic segmentation, the accident vehicle can be accurately segmented from the color image. The segmented color image is aligned with the first depth image through an adaptive stereo correction algorithm to ensure the spatial consistency of color and depth information. Through precise image segmentation and depth information fusion, the data processing process is optimized, thereby improving the efficiency of the overall measurement process.

[0100] Preferably, a panoramic segmentation algorithm is used to perform panoramic segmentation on each of the color images. The panoramic segmentation algorithm processes the input color image through a pre-trained deep neural network model, firstly performs semantic segmentation to generate a probability map of the category to which each pixel belongs, solves the classification problem of the background and foreground, and ensures the accurate division of different category areas. Then, the instance segmentation technology is used to further refine the segmentation result to ensure that each independent target object can be accurately separated, solves the problem of distinguishing between targets, and can effectively identify and locate each component, especially for the vehicle parts that are severely deformed or damaged. Through the scene segmentation algorithm, even in the case of complex background or severe changes in the target morphology, the various parts of the accident vehicle can be effectively identified and located, thereby improving the accuracy and robustness of accident vehicle detection.

[0101] Preferably, an adaptive stereo correction (ASR) algorithm is used to align the segmented color image with the depth image and extract the depth information of the vehicle body. The correction transformation matrix is ​​dynamically calculated according to the internal and external parameters of the camera and real-time data to ensure that the color image and the depth image are aligned in space.

[0102] like Figure 4 As shown, in this embodiment, the S4 specifically includes the following steps:

[0103] S41: Converting each of the second depth images into each of the three-dimensional point cloud data according to the built-in parameters of the depth camera;

[0104] S42: extracting a plurality of feature points from each of the three-dimensional point cloud data;

[0105] S43: selecting a plurality of matching points from each of the feature points, calculating a rigid transformation between the matching points, and obtaining a transformation matrix through multiple iterations;

[0106] S44: Optimizing the transformation matrix until the registration error between the matching points is lower than a set threshold or reaches a preset number of iterations, and obtaining a final change matrix;

[0107] S45: splicing the three-dimensional point cloud data according to the final change matrix to obtain the three-dimensional point cloud data of the accident vehicle;

[0108] S46: removing outliers from the three-dimensional point cloud data of the accident vehicle to obtain the outer contour three-dimensional point cloud data, and then fitting a geometric surface model of the vehicle body according to the outer contour three-dimensional point cloud data.

[0109] The second depth image is converted into three-dimensional point cloud data through the built-in parameters of the depth camera, ensuring the consistency of the spatial coordinates of the data with the actual scene; the three-dimensional point cloud data is matched using the progressive consistent sampling algorithm and the iterative closest point algorithm, which improves the efficiency and accuracy of data processing and ensures the consistency and accuracy of the point cloud data; the first matching is performed using the progressive consistent sampling algorithm, and then the result is matched a second time based on the iterative closest point algorithm, which enhances the registration accuracy of the point cloud data under different viewing angles; the point cloud data from multiple angles are spliced ​​into a complete three-dimensional point cloud data of the outer contour of the accident vehicle, providing comprehensive geometric information of the accident vehicle.

[0110] Preferably, the feature extraction method includes a fast point feature histogram (FPFH) algorithm and a normal distribution transform (NDT) algorithm. These feature points can capture the local geometric structure of the point cloud and provide a reliable matching basis for subsequent registration.

[0111] Preferably, the Progressive Consistent Sampling (PROSAC) algorithm is used to select matching points from the extracted feature points, calculate the rigid transformation (rotation and translation) between the matching points, and select the optimal transformation matrix through multiple iterations to achieve coarse alignment of the point cloud, wherein the PROSAC algorithm can more effectively exclude outliers and improve the robustness and efficiency of the alignment.

[0112] Preferably, an iterative closest point (ICP) algorithm is used for fine registration. The ICP algorithm iteratively finds the closest point pair and gradually optimizes the transformation matrix.

[0113] Preferably, the registration error between matching points is lower than a set threshold of 1 cm, and the preset number of iterations is 50 times, so as to avoid the algorithm falling into a local optimal solution and improve the efficiency and stability of the registration.

[0114] More preferably, statistical filtering is used to remove outliers from the spliced ​​point cloud, where the number of neighborhood points is taken as 50. In order to fit the geometric surface model of the vehicle body from the point cloud, a Poisson surface reconstruction algorithm can be used, which can effectively fit the geometric surface model of the vehicle body from the point cloud data and ensure the smoothness and accuracy of the model.

[0115] like Figure 5 As shown, in this embodiment, S5 specifically includes the following steps:

[0116] S51: clustering the outer contour three-dimensional point cloud data to extract three-dimensional point cloud data describing the body of the accident vehicle;

[0117] S52: Extracting the outline of the accident vehicle from the three-dimensional point cloud data describing the body of the accident vehicle by using a boundary extraction method.

[0118] Clustering the three-dimensional point cloud data can effectively separate the vehicle-related point cloud data from the background or irrelevant data, thereby improving the accuracy of data processing; filtering out the cluster containing the vehicle body from the clustering results can accurately locate the point cloud data of the accident vehicle; and extracting the outline of the accident vehicle from the filtered vehicle body clusters through the boundary extraction method. This step ensures the accurate recognition of the vehicle's outer contour.

[0119] Preferably, the clustering method adopts the density-based clustering with noise (HDBSCAN) algorithm, which is a density-based hierarchical clustering algorithm that can better handle data sets with different densities and can automatically determine the number of clusters. It performs well when processing data sets with different densities; the boundary extraction method includes one or more algorithms of the convex hull algorithm and the rolling ball method (Alpha Shapes algorithm).

[0120] like Figure 6 As shown, in this embodiment, S6 specifically includes the following steps:

[0121] S61: Convert the three-dimensional point cloud data describing the body of the accident vehicle into a matrix representation to obtain a three-dimensional point cloud data matrix, wherein each row in the three-dimensional point cloud data matrix represents the coordinates of each point;

[0122] S62: decomposing the three-dimensional point cloud data matrix into a low-rank matrix and a sparse matrix, obtaining the main directions of the three-dimensional point cloud data by singular value decomposition of the low-rank matrix, and then establishing a three-dimensional coordinate system and obtaining the three main axis directions of the three-dimensional coordinate system;

[0123] S63: construct multiple directed bounding boxes in each of the main axis directions based on the edge contour of the accident vehicle and the minimum volume directed bounding box algorithm, obtain the total length of the multiple directed bounding boxes in each of the main axis directions, and then obtain the outer dimensions of the accident vehicle.

[0124] The three-dimensional point cloud data matrix is ​​decomposed into a low-rank matrix and a sparse matrix using singular value decomposition technology, and the main directions of the three-dimensional data are effectively extracted through singular value decomposition of the low-rank matrix, thereby improving the efficiency of data processing; by constructing multiple directed bounding boxes in each main axis direction and using the minimum volume directed bounding box algorithm, the box that is closest to the vehicle contour can be found to obtain the smallest volume, thereby improving the efficiency and accuracy of volume calculation; directly constructing a directed bounding box on the three-dimensional point cloud data simplifies the data processing process and improves the speed and efficiency of data processing.

[0125] Preferably, a robust principal component analysis method is used to decompose the three-dimensional point cloud data matrix into a low-rank matrix and a sparse matrix, which can better handle noise and outliers in the data, thereby providing a more accurate main direction estimation; a minimum volume directed bounding box algorithm is used to construct a directed bounding box, and the volume of the bounding box is minimized by adjusting the rotation and size of the bounding box, so as to better fit the point cloud data. This algorithm can generate a more compact and accurate bounding box, reduce unnecessary gaps and redundancy, and improve the accuracy and robustness of the model.

[0126] In summary, if Figure 1 to Figure 7 As shown, a method for measuring the outer dimensions of an accident vehicle based on a depth camera comprises the following steps:

[0127] S1: Continuously collect contour images of the accident vehicle at multiple angles through a depth camera, perform integrity detection and overlapping area detection on the collected contour images, and if the overlapping area is not less than a set value, the total coverage angle is 360°, and the measured angle difference is less than a preset angle, then enter step S2; otherwise, re-acquire the image, and the contour image includes the original color image and the original depth image;

[0128] S2: Preprocessing the outline image at each angle respectively, obtaining the color image and the first depth image at each angle of the accident vehicle, performing noise reduction processing on the original depth image, and performing data cleaning, filtering out data points with a depth value of 0 or invalid in the original depth image, so as to obtain the first depth image at each angle of the accident vehicle; performing image enhancement on the original color image, and performing distortion correction, so as to obtain the color image at each angle of the accident vehicle;

[0129] S3: performing panoramic segmentation on each of the color images to segment out the accident vehicle part; aligning each of the segmented color images with each of the first depth images, and performing mask processing on the segmented accident vehicles in each of the color images to obtain a depth value of a corresponding position of the accident vehicle, thereby obtaining each of the corresponding second depth images;

[0130] S4: converting the second depth images at multiple angles into three-dimensional point cloud data, and registering and splicing the three-dimensional point cloud data to obtain three-dimensional point cloud data of the outer contour of the accident vehicle;

[0131] S5: clustering and screening the three-dimensional point cloud data of the outer contour of the accident vehicle, and extracting the outer contour of the accident vehicle by using a boundary extraction method;

[0132] S6: Establishing the stereo coordinate axes of the preprocessed three-dimensional point cloud data based on the edge contour of the accident vehicle, obtaining the length of the accident vehicle along the direction of each stereo coordinate axis, and then obtaining the outer dimensions of the accident vehicle.

[0133] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for measuring the outer dimensions of an accident vehicle based on a depth camera, characterized in that: The following steps are involved: S1: Continuously collect the outline images of the accident vehicle from multiple angles through the depth camera; S2: Preprocessing the outline images at various angles respectively to obtain color images and first depth images of the accident vehicle at various angles; S3: performing segmentation processing on the color image at each angle, and mapping the segmented color image to the first depth image at the same angle to generate a second depth image; S4: converting the second depth images at multiple angles into three-dimensional point cloud data, and registering and splicing the three-dimensional point cloud data to obtain three-dimensional point cloud data of the outer contour of the accident vehicle; S5: clustering and screening the three-dimensional point cloud data of the outer contour of the accident vehicle, and extracting the outer contour of the accident vehicle by using a boundary extraction method; S6: Establishing the stereo coordinate axes of the preprocessed three-dimensional point cloud data based on the edge contour of the accident vehicle, obtaining the length of the accident vehicle along the direction of each stereo coordinate axis, and then obtaining the outer dimensions of the accident vehicle.

2. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The S1 specifically includes: S11: using a depth camera to perform surround capture of the accident vehicle according to a preset interval angle, and obtaining contour images of the accident vehicle at multiple angles; S12: Perform integrity check and overlapping area check on the collected contour image. If the overlapping area is not less than the set value, the total coverage angle is 360°, and the measured angle difference is less than the preset angle, then enter step S2; otherwise, re-collect the image.

3. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 2, characterized in that: The integrity detection comprises the following steps: sorting the contour images according to the acquisition angle to obtain the total coverage angle of all the contour images; and calculating the angle difference between two contour images with adjacent angles.

4. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 2, characterized in that: The overlapping area detection comprises the following steps: Extract feature points from each of the outline images, perform feature matching on two angularly adjacent outline images, remove mismatched feature points, and generate a plurality of homography matrices between two angularly adjacent outline images; obtain the intersection area of ​​the bounding boxes of two angularly adjacent outline images according to each of the homography matrices, and calculate the ratio of the intersection area of ​​the bounding boxes to the total area of ​​the outline image.

5. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The outline image at each angle includes an original depth image and an original color image.

6. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 5, characterized in that: The S2 specifically includes the following steps: Performing noise reduction processing on the original depth image and data cleaning, filtering out data points with a depth value of 0 or invalid in the original depth image, so as to obtain the first depth image of each angle of the accident vehicle; The original color image is enhanced and distortion corrected to obtain the color image of the accident vehicle at various angles.

7. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The S3 specifically includes the following steps: S31: performing panoramic segmentation on each of the color images to segment out the accident vehicle part; S32: Aligning each of the segmented color images with each of the first depth images, and performing mask processing on the accident vehicle segmented in each of the color images to obtain a depth value of a corresponding position of the accident vehicle, and thereby obtaining the corresponding second depth images.

8. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The S4 specifically comprises the following steps: S41: Converting each of the second depth images into each of the three-dimensional point cloud data according to the built-in parameters of the depth camera; S42: extracting a plurality of feature points from each of the three-dimensional point cloud data; S43: selecting a plurality of matching points from each of the feature points, calculating a rigid transformation between the matching points, and obtaining a transformation matrix through multiple iterations; S44: Optimizing the transformation matrix until the registration error between the matching points is lower than a set threshold or reaches a preset number of iterations, and obtaining a final change matrix; S45: splicing the three-dimensional point cloud data according to the final change matrix to obtain the three-dimensional point cloud data of the accident vehicle; S46: removing outliers from the three-dimensional point cloud data of the accident vehicle to obtain the outer contour three-dimensional point cloud data, and then fitting a geometric surface model of the vehicle body according to the outer contour three-dimensional point cloud data.

9. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The S5 specifically includes the following steps: S51: clustering the outer contour three-dimensional point cloud data to extract three-dimensional point cloud data describing the body of the accident vehicle; S52: Extracting the outline of the accident vehicle from the three-dimensional point cloud data describing the body of the accident vehicle by using a boundary extraction method.

10. The method for measuring the outer dimensions of an accident vehicle based on a depth camera according to claim 1, characterized in that: The S6 specifically comprises the following steps: S61: Convert the three-dimensional point cloud data describing the body of the accident vehicle into a matrix representation to obtain a three-dimensional point cloud data matrix, wherein each row in the three-dimensional point cloud data matrix represents the coordinates of each point; S62: decomposing the three-dimensional point cloud data matrix into a low-rank matrix and a sparse matrix, obtaining the main directions of the three-dimensional point cloud data by singular value decomposition of the low-rank matrix, and then establishing a three-dimensional coordinate system and obtaining the three main axis directions of the three-dimensional coordinate system; S63: Based on the edge contour of the accident vehicle and the minimum volume directed bounding box algorithm, multiple directed bounding boxes are constructed in each of the main axis directions, and the total length of the multiple directed bounding boxes in each of the main axis directions is obtained, thereby obtaining the outer dimensions of the accident vehicle.

Images