An intelligent wheel axle recognition method and device based on laser + visual AI technology

Abstract

The present invention relates to the field of image data processing technology, and in particular to an intelligent axle identification method and device based on laser + visual AI technology. The method comprises the following steps: scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data; collecting vehicle images by a high-definition camera to obtain vehicle image data; extracting features from the vehicle image data to obtain vehicle feature data; performing multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain vehicle multi-dimensional fusion data for intelligent axle identification operations. The present invention improves the ability to identify vehicles, makes the processing capability of the device more robust and reliable, can adapt to different environments and provide reliable data support to increase data granularity and improve data accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to an intelligent wheel axle recognition method and device based on laser + visual AI technology. Background Art

[0002] The intelligent axle recognition method is a technology used to identify and monitor axles. It is usually used in railways, road traffic, industrial equipment and other fields. It aims to use various sensors, computer vision and machine learning technologies to automatically detect and track the position, status and movement information of axles. At present, the rapid and accurate identification of vehicle axles and models is of great significance in scenarios such as highway entrances and exits, non-stop inspection stations and freight source companies. However, traditional detection methods have problems such as low accuracy, high interference and poor environmental adaptability. Summary of the invention

[0003] In order to solve the above-mentioned technical problems, the present invention proposes an intelligent wheel axle recognition method and device based on laser + visual AI technology to solve at least one of the above-mentioned technical problems.

[0004] The present application provides an intelligent wheel axle recognition method based on laser + visual AI technology, the method comprising:

[0005] S1, scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data;

[0006] S2, collecting vehicle images through a high-definition camera to obtain vehicle image data;

[0007] S3, extracting features from the vehicle image data to obtain vehicle feature data;

[0008] S4. Perform multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle for performing intelligent wheel axle recognition operation.

[0009] In the present invention, by combining laser dot cloud data and vehicle image data, the method can provide more comprehensive information, thereby achieving high-precision identification of vehicle axles. Laser dot cloud data provides information on three-dimensional shape and size, while vehicle image data provides visual features, and the fusion of the two improves recognition accuracy. This method does not require the vehicle to stop or perform special operations, and can perform axle identification while the vehicle is driving, thereby improving detection efficiency, and is particularly suitable for scenes such as highway entrances and exits and non-stop inspection stations. By performing multi-dimensional data fusion of laser dot cloud data and vehicle image data, information from different sensors can be comprehensively utilized, thereby improving the comprehensive understanding and identification capabilities of the vehicle, making the device more robust and reliable, able to adapt to different environments, and providing reliable data support to increase data granularity and improve data accuracy.

[0010] Optionally, scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data includes:

[0011] S11, controlling the laser sensor to emit laser light to scan the vehicle to be measured and receiving reflected light to obtain laser scanning data;

[0012] S12, generating laser point cloud data from the laser scanning data to obtain laser point cloud data;

[0013] S13, performing noise reduction processing on the laser point cloud data to obtain point cloud noise reduction data;

[0014] S14, down-sampling the point cloud denoised data to obtain point cloud down-sampled data;

[0015] S15, performing point cloud registration on the point cloud using data to obtain laser point cloud data.

[0016] In the present invention, the vehicle is scanned by a laser sensor, and laser scanning data is first obtained, and then converted into laser point cloud data. The beneficial effect of this step is that it provides three-dimensional information of the vehicle surface, including body shape, wheel position and size, etc., providing basic data for subsequent identification. After obtaining the point cloud data, noise reduction processing helps to remove unnecessary data points caused by sensor or environmental noise, thereby improving data quality. Point cloud downsampling can reduce the amount of data and reduce computing costs, while retaining the main features, which helps to improve computing efficiency. Laser point cloud data usually comes from scans at different perspectives or times, and point cloud registration is to collect these data into a global coordinate system to obtain more complete three-dimensional information. This is beneficial for accurately identifying the number of wheels, the number of axles, and the model of the vehicle.

[0017] Optionally, the laser scanning data includes first laser scanning data and second laser scanning data, and the laser sensor is controlled to emit laser to scan the vehicle to be measured and receive reflected light to obtain the laser scanning data, including:

[0018] Controlling the laser sensor installed at the bottom to emit a first laser to scan the vehicle to be measured and receive a first reflected light to obtain the first laser scanning data;

[0019] The laser sensor installed on the side is controlled to emit a second laser to scan the vehicle to be measured and receive a second reflected light to obtain the second laser scanning data, wherein the scanning angle of the first laser is smaller than the scanning angle of the second laser, the laser power corresponding to the first laser is larger than the laser power corresponding to the second laser, and the adopted density data corresponding to the first laser is larger than the adopted density data of the second laser.

[0020] The present invention uses bottom and side laser sensors to obtain laser scanning data of different angles and attributes. The bottom sensor can provide more information about the bottom of the vehicle, while the side sensor can provide side information. This multi-sensor fusion helps to integrate information from different perspectives and improve the accuracy of vehicle identification. The laser power corresponding to the first laser is greater than that of the second laser, which means that the first laser performs reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data during measurement, and obtains reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data with higher brightness and sensitivity, which can improve the measurement performance under low light conditions and ensure data quality. The density data corresponding to the first laser is greater than that of the second laser, which indicates that the first laser can provide denser data points, thereby describing the characteristics of the vehicle in more detail, which helps to improve the accuracy and details of vehicle identification.

[0021] Optionally, the generating laser point cloud data from the laser scanning data to obtain the laser point cloud data includes:

[0022] S121, extracting reflection intensity data from the laser scanning data to obtain reflection intensity data;

[0023] S122, performing reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data, to obtain reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data respectively;

[0024] S123, using a preset reflection intensity material classification model to identify the reflection intensity mean data, the reflection intensity standard deviation data, and the reflection intensity gradient data to obtain reflection intensity material data, wherein the reflection intensity material classification model is constructed and generated by using a support vector machine algorithm using preset historical reflection intensity mean data, historical reflection intensity standard deviation data, historical reflection intensity gradient data, and corresponding material label data;

[0025] S124, marking the laser scanning data using the reflection intensity material data corresponding to the reflection intensity data to obtain laser scanning marking data;

[0026] S125, generating point cloud data from the laser scanning mark data to obtain laser point cloud data;

[0027] The step of extracting reflection intensity data from the laser scanning data to obtain reflection intensity data includes:

[0028] S1231, performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data;

[0029] S1232, calculating the normal direction of the local point cloud surface fitting data to obtain local point cloud normal direction data;

[0030] S1233, performing curvature calculation on the local point cloud surface fitting data to obtain local point cloud curvature data;

[0031] S1234, dividing the laser scanning data according to the local point cloud normal direction data and the local point cloud curvature data to obtain a laser scanning sub-data set;

[0032] S1235. Extract reflection intensity data from the laser scanning sub-dataset to obtain reflection intensity data.

[0033] In the present invention, by analyzing the mean, standard deviation and gradient of the reflection intensity data, combined with the preset material classification model, the material can be identified, which helps to determine the structural material of the vehicle surface, such as metal, plastic, rubber, etc., which is very important for further analysis and identification of the vehicle. By fitting the local point cloud surface, calculating the normal direction and curvature, the local feature information of each point can be obtained. These features help to describe the geometric structure of the surface in more detail, including curvature, normal direction, etc. This can help to more accurately identify the surface of different parts of the vehicle. Marking the reflection intensity material data into the laser point cloud data can make each point carry information about its surface material, which can be used for subsequent tasks such as vehicle identification, object detection, etc. Segmenting the laser scanning data according to the local feature data can refine the data into smaller subsets, which helps to understand the characteristics of the vehicle surface in more detail. Using the support vector machine algorithm to construct a reflection intensity material classification model can efficiently identify the material, while improving the robustness and scalability of the system. Through multi-dimensional data fusion and reflection intensity material recognition, it is helpful to more accurately analyze the vehicle surface and improve the performance and reliability of the intelligent wheel axle recognition system.

[0034] Optionally, performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data includes:

[0035] Performing random point sampling and cluster calculation on the laser scanning data to obtain random point sampling data and laser scanning cluster feature data;

[0036] Screening the random point adoption data and preset threshold distance data to obtain random point screening data;

[0037] Performing density calculation on the random point screening data and the preset threshold radius data to obtain first laser scanning density data;

[0038] Performing density calculation on the laser scanning cluster feature data to obtain second laser scanning density data;

[0039] Performing multi-scale local neighborhood mapping according to the first laser scanning density data and the second laser scanning density data to obtain first multi-scale local neighborhood data and second multi-scale local neighborhood data, wherein the multi-scale segmentation mapping is generated by mapping through preset scale space pyramid data, and the preset scale space pyramid data is neighborhood radius threshold data corresponding to different laser scanning density threshold data;

[0040] Extracting local neighborhood data from the laser scanning data according to the first multi-scale local neighborhood data, the second multi-scale local domain data, and the random point screening data to obtain local neighborhood data;

[0041] Surface fitting parallel calculation is performed on the local neighborhood data to obtain local point cloud surface fitting data.

[0042] In the present invention, through multi-scale local neighborhood mapping, point cloud data can be analyzed at different scales, which helps to better capture the features and structures in the data. Different scale levels correspond to different features or objects, thereby providing more comprehensive information and more fine-grained data operations to provide accurate and reliable data support. Dividing point cloud data into local neighborhoods helps to extract local features near each point, helps to describe the geometric characteristics of the surface in more detail, and reduces computational complexity. By calculating laser scanning density data, data points can be divided into different density clusters, which helps to separate different structures in the point cloud and better understand the uniformity and sparsity of the data. Through parallel calculation of surface fitting, the fitting surface of each local neighborhood can be obtained, and these fitting surfaces can be used for subsequent analysis and processing, such as feature extraction, object recognition, etc.

[0043] Optionally, downsampling the point cloud denoised data to obtain point cloud downsampled data includes:

[0044] Performing density clustering calculation on the point cloud denoising data to obtain point cloud density clustering data;

[0045] Perform density cluster standard deviation calculation and density cluster density calculation on the point cloud density clustering data to obtain a density cluster standard deviation data set and a density cluster density data set;

[0046] Performing density difference processing on the density standard deviation data of the density standard deviation data and the density cluster density data of the density cluster density data set to obtain density difference data, wherein the density difference data includes uniform density data and non-uniform density data;

[0047] When it is determined that the density difference data is density uniform data, voxel grid downsampling processing is performed on the point cloud denoising data to obtain the first point cloud downsampling data;

[0048] When it is determined that the density difference data is uniform density data, the point cloud denoising data is statistically downsampled according to the density cluster standard deviation data set and the density cluster density data set to obtain the second point cloud downsampled data, wherein the point cloud downsampled data includes the first point cloud downsampled data and the second point cloud downsampled data.

[0049] In the present invention, by using density clustering calculation, the method can perceive the density distribution in the point cloud data, and judge the density uniformity of the point cloud according to the density cluster standard deviation and density cluster density data, so that the downsampling process is more intelligent, and the downsampling strategy is determined according to the actual density of the point cloud. When the point cloud density is relatively uniform, voxel grid downsampling can effectively retain the main features in the point cloud and reduce the amount of data, which helps to reduce the computational complexity in subsequent processing while maintaining key information. When the point cloud density is uneven, statistical downsampling can be used to capture density changes in more detail, ensuring that more points are retained in areas with higher density, thereby better reflecting the characteristics of the actual scene.

[0050] Optionally, the point cloud adopted data includes first point cloud adopted data and second point cloud adopted data, the first point cloud adopted data and the second point cloud adopted data are point cloud sampling data corresponding to laser adopted data collected from the same vehicle by different sensors or at different angles, and performing point cloud registration on the point cloud adopted data to obtain laser dot cloud data includes:

[0051] Extracting description points from the data used in the first point cloud and the data used in the second point cloud to obtain description point data;

[0052] Perform feature point matching on the data used in the first point cloud and the data used in the second point cloud according to the description point data to obtain feature point matching data;

[0053] generating preliminary transformation data according to the feature point matching data;

[0054] Using the preliminary transformed data, performing least squares iterative optimization on the data used in the first point cloud and the data used in the second point cloud to obtain optimized transformed data;

[0055] Point cloud fusion is performed on the data used in the first point cloud and the data used in the second point cloud according to the optimized transformation data to obtain laser point cloud data.

[0056] In the present invention, by simultaneously using the first point cloud down-sampled data and the second point cloud down-sampled data collected by different sensors, the method allows for more comprehensive capture of vehicle or scene information, and this multi-sensor data fusion helps to improve the accuracy and completeness of the data. Using description point extraction and feature point matching, the correspondence between the first point cloud down-sampled data and the second point cloud down-sampled data can be determined. By applying the least squares iterative optimization, the alignment quality of the point cloud can be further improved, ensuring that the point cloud data collected by different sensors or at different angles maintain an accurate spatial relationship after fusion. The method fuses the first point cloud down-sampled data and the second point cloud down-sampled data into a unified laser dot cloud data, which is helpful for subsequent processing and analysis, such as intelligent wheel axle recognition.

[0057] Optionally, the vehicle feature data includes vehicle color feature data, vehicle texture feature data, vehicle shape feature data and vehicle local feature data, and the feature extraction of the vehicle image data to obtain the vehicle feature data includes:

[0058] Extracting vehicle color features from the vehicle image data to obtain vehicle color feature data;

[0059] Extracting vehicle texture features from the vehicle image data to obtain the vehicle texture feature data;

[0060] Extracting vehicle shape features from the vehicle image data to obtain vehicle shape feature data;

[0061] The vehicle local feature extraction is performed on the vehicle image data to obtain the vehicle local feature data.

[0062] By extracting the vehicle color, texture, shape and local features in the present invention, the method can more comprehensively describe the characteristics of the vehicle to be measured, which helps to improve the accuracy of intelligent wheel axle recognition because it includes not only point cloud data but also visual information. Fusion of different types of vehicle feature data can provide more representative feature vectors, which helps to distinguish different vehicles. Covering multiple feature types makes the method applicable to vehicles of different types and styles, and the vehicle can be identified by this method regardless of its color, texture, shape or local features.

[0063] Optionally, performing multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fused vehicle data includes:

[0064] Performing data alignment on the laser dot cloud data and the vehicle feature data to obtain dot vehicle feature alignment data;

[0065] Performing self-attention calculation on the dot matrix vehicle feature alignment data to obtain aligned self-attention data;

[0066] The aligned self-attention data is subjected to feature weighted fusion processing to obtain vehicle multi-dimensional fusion data; or, the aligned self-attention data is subjected to feature connection to obtain vehicle multi-dimensional fusion data.

[0067] In the present invention, the laser dot cloud data and the vehicle feature data are fused to fully integrate the information from different sensor sources. This helps to improve the accuracy of vehicle recognition because different data sources provide different perspectives and features. Through self-attention calculation, the system can better focus on the features related to the recognition task and automatically learn the importance between different features, which helps to reduce the impact of irrelevant or noise features and improve the stability of recognition. In the process of multi-dimensional data fusion, the self-attention data is aligned for feature weighted fusion, and each feature can be weighted according to its importance, thereby improving the overall fusion effect. By fusing multiple feature data and automatically learning feature importance, this method can improve the accuracy of vehicle recognition and help to more accurately identify and track target vehicles.

[0068] Optionally, the present application further provides an intelligent wheel axle recognition device based on laser + visual AI technology, which is used to execute the intelligent wheel axle recognition method based on laser + visual AI technology as described above, and the intelligent wheel axle recognition device based on laser + visual AI technology includes:

[0069] The laser scanning module is used to scan the vehicle to be measured through a laser sensor to obtain laser dot cloud data;

[0070] The vehicle image acquisition module is used to acquire vehicle images through a high-definition camera to obtain vehicle image data;

[0071] A feature extraction module, used to extract features from the vehicle image data to obtain vehicle feature data;

[0072] The multi-dimensional data fusion module is used to perform multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle for performing intelligent wheel axle recognition operations.

[0073] The purpose of the present invention is that the method first obtains laser dot cloud data through a laser sensor, which can provide accurate information about the external structure and shape of the vehicle. At the same time, the vehicle image data is obtained through a high-definition camera, which can capture the color, texture, logo and other visual information of the vehicle. The two different data sources are fused to achieve comprehensive utilization of information, which is helpful for a more comprehensive understanding and identification of the vehicle. The laser dot cloud data provides high-resolution vehicle external information, including the geometric features of the axle. By extracting features from these data, the position and shape of the axle of the vehicle can be accurately identified, providing accurate and reliable data support for the accurate positioning and identification of the axle, especially in traffic monitoring, intelligent transportation systems and vehicle management. The vehicle image data provides visual information such as color, texture and logo, which helps to further verify the accuracy of the laser data, as well as the identification and classification of the vehicle. For example, the color and logo of the vehicle can be used to distinguish different types of vehicles, such as trucks, cars or buses. In step S4, the method merges the laser dot cloud data and the vehicle image data into a unified data set through multidimensional data fusion. Multidimensional data fusion helps to integrate data from different sources into a comprehensive representation, improving the comprehensive understanding and identification accuracy of the vehicle axle. The data generated by the present invention can realize intelligent wheel axle identification operation. Through this method, accurate and reliable non-contact data support can be provided for the automatic detection and identification of vehicle wheel axles, and it has broad application prospects in the fields of traffic monitoring, intelligent transportation systems, road safety and vehicle management. The system can intelligently identify the position, number and shape of wheel axles based on the comprehensive information of laser and visual data, providing a more powerful tool for traffic management and monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0075] Figure 1 A flowchart showing a method for intelligent wheel axle recognition based on laser + visual AI technology according to an embodiment of the present invention is shown;

[0076] Figure 2 A flowchart showing the steps of a method for generating laser dot cloud data according to an embodiment is shown;

[0077] Figure 3 A flowchart showing the steps of a method for generating laser point cloud data according to an embodiment is shown;

[0078] Figure 4 A flowchart showing the steps of a reflection intensity data extraction method according to an embodiment is shown;

[0079] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0080] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0081] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0082] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0083] See also Figures 1 to 4 , the present application provides an intelligent wheel axle recognition method based on laser + visual AI technology, the method comprising:

[0084] S1, scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data;

[0085] In one embodiment of the present application, a laser sensor is used to perform a 360-degree scan around the vehicle to generate three-dimensional laser point cloud data, which includes information such as the vehicle's external structure, axles, wheels, etc.

[0086] In one embodiment of the present application, multiple laser sensors are used for scanning to obtain higher resolution point cloud data to improve the recognition accuracy of the wheel axle.

[0087] S2, collecting vehicle images through a high-definition camera to obtain vehicle image data;

[0088] In one embodiment of the present application, a high-definition camera is used to capture the front, rear, left, and right view images of the vehicle to ensure that the appearance features of the vehicle are captured from different angles.

[0089] In one embodiment of the present application, multiple cameras are used to obtain vehicle image data with an all-round perspective to increase the amount of information about the vehicle's appearance.

[0090] S3, extracting features from the vehicle image data to obtain vehicle feature data;

[0091] In one embodiment of the present application, computer vision technology is used to extract color features of vehicle images, including main colors and color histograms.

[0092] In one embodiment of the present application, an image texture analysis method is used to extract texture features of a vehicle image, such as a local binary pattern (LBP) or a gray level co-occurrence matrix (GLCM).

[0093] In one embodiment of the present application, deep learning technology, such as convolutional neural network (CNN), is applied to extract shape features or local features of the vehicle from the image.

[0094] S4. Perform multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle for performing intelligent wheel axle recognition operation.

[0095] In one embodiment of the present application, the laser point cloud data and the vehicle image data are aligned, for example, the 3D coordinates in the point cloud data are mapped to the image coordinate system.

[0096] In one embodiment of the present application, self-attention calculation is used to weight information from different data sources to emphasize features with high recognition.

[0097] In one embodiment of the present application, a feature weighted fusion method is used to combine the features of point cloud data and image data according to the importance of different features.

[0098] In the present invention, by combining laser dot cloud data and vehicle image data, the method can provide more comprehensive information, thereby achieving high-precision identification of vehicle axles. Laser dot cloud data provides information on three-dimensional shape and size, while vehicle image data provides visual features. The fusion of the two improves recognition accuracy. This method does not require the vehicle to stop or perform special operations. It can identify axles while the vehicle is driving, which improves detection efficiency. It is particularly suitable for scenes such as highway entrances and exits and non-stop inspection stations. By performing multi-dimensional data fusion of laser dot cloud data and vehicle image data, information from different sensors can be comprehensively utilized, which improves the comprehensive understanding and recognition capabilities of the vehicle, making the system more robust and reliable.

[0099] Optionally, scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data includes:

[0100] S11, controlling the laser sensor to emit laser light to scan the vehicle to be measured and receiving reflected light to obtain laser scanning data;

[0101] In one embodiment of the present application, a laser sensor is installed under the vehicle to scan the vehicle to be measured at a fixed angle and scanning frequency, and the distance and angle information of each scanning point are measured. The sensor emits a laser beam and receives reflected light to generate laser scanning data.

[0102] S12, generating laser point cloud data from the laser scanning data to obtain laser point cloud data;

[0103] In one embodiment of the present application, the laser scanning data is processed and the distance and angle information is converted into a point cloud format. Each point represents the location where the laser beam hits and contains X, Y, Z coordinate information, as well as a reflection intensity value. Laser point cloud data is a data set consisting of discrete points, each point has three-dimensional coordinates (X, Y, Z), and usually also includes reflection intensity information and other attributes. These points are discrete and there is no specific order of arrangement.

[0104] S13, performing noise reduction processing on the laser point cloud data to obtain point cloud noise reduction data;

[0105] In one embodiment of the present application, a filtering algorithm, such as statistical filtering, Gaussian filtering, or moving average filtering, is used to remove noise points in the laser point cloud data to obtain cleaner point cloud data.

[0106] In one embodiment of the present application, a local outlier detection method, such as Local Outlier Factor (LOF) or Radius-Based Outlier Detection (ROD), can detect outliers based on local neighborhood information of the point. The local outlier detection method is used to detect and remove local discrete points of the laser point cloud data to obtain point cloud denoised data.

[0107] In one embodiment of the present application, whether a point is a noise point is determined based on the density of the point in the point cloud, and points with a density lower than a threshold are regarded as noise and removed.

[0108] S14, down-sampling the point cloud denoised data to obtain point cloud down-sampled data;

[0109] In one embodiment of the present application, a downsampling algorithm, such as voxel grid downsampling or statistical downsampling, is used to reduce the points in the point cloud data to a lower density to reduce the amount of data and improve processing efficiency, thereby obtaining point cloud downsampled data.

[0110] S15, performing point cloud registration on the point cloud using data to obtain laser point cloud data.

[0111] In one embodiment of the present application, a point cloud registration algorithm, such as an iterative closest point (ICP) algorithm, is used to align point cloud data collected by different scans or sensors to obtain the entire laser point cloud data, in which the relative position relationship between the point clouds is determined. Laser point cloud data is a set of laser points arranged in a regular grid or matrix form, which is usually used to represent dense information of a surface or scene. These data are usually processed for tasks such as surface reconstruction or image generation.

[0112] In the present invention, the vehicle is scanned by a laser sensor, and laser scanning data is first obtained, and then converted into laser point cloud data. The beneficial effect of this step is that it provides three-dimensional information of the vehicle surface, including body shape, wheel position and size, etc., providing basic data for subsequent identification. After obtaining the point cloud data, noise reduction processing helps to remove unnecessary data points caused by sensor or environmental noise, thereby improving data quality. Point cloud downsampling can reduce the amount of data and reduce computing costs, while retaining the main features, which helps to improve computing efficiency. Laser point cloud data usually comes from scans at different perspectives or times, and point cloud registration is to collect these data into a global coordinate system to obtain more complete three-dimensional information. This is beneficial for accurately identifying the number of wheels, the number of axles, and the model of the vehicle.

[0113] Optionally, the laser scanning data includes first laser scanning data and second laser scanning data, and the laser sensor is controlled to emit laser to scan the vehicle to be measured and receive reflected light to obtain the laser scanning data, including:

[0114] Controlling the laser sensor installed at the bottom to emit a first laser to scan the vehicle to be measured and receive a first reflected light to obtain the first laser scanning data;

[0115] In one embodiment of the present application, a bottom laser sensor is installed so that it is located at the bottom of the vehicle, emitting a laser beam and receiving reflected light. The first laser scanning data is generated by the bottom sensor, and the scanning angle is relatively small, which is suitable for obtaining information about the bottom of the vehicle and the wheels. The laser power of the first laser is set high to ensure that sufficient reflected signals can be obtained at a long distance from the bottom of the vehicle. The first laser uses density data to describe the structure of the bottom of the vehicle.

[0116] The laser sensor installed on the side is controlled to emit a second laser to scan the vehicle to be measured and receive a second reflected light to obtain the second laser scanning data, wherein the scanning angle of the first laser is smaller than the scanning angle of the second laser, the laser power corresponding to the first laser is larger than the laser power corresponding to the second laser, and the adopted density data corresponding to the first laser is larger than the adopted density data of the second laser.

[0117] In one embodiment of the present application, a side laser sensor is installed so that it is located on the side of the vehicle, emitting a laser beam and receiving reflected light. The second laser scanning data is generated by the side sensor, and the scanning angle is relatively large, which is suitable for obtaining detailed information on the side of the vehicle and the external structure. The laser power of the second laser is set moderately to adapt to the reflected signal at a closer distance. The density data of the second laser is used to describe the structure of the side of the vehicle.

[0118] In one embodiment of the present application, the first laser scanning data: scanning angle: 30 degrees, laser power: 1000 milliwatts, density data: 200 points / square meter, the second laser scanning data: scanning angle: 60 degrees, laser power: 800 milliwatts, density data: 150 points / square meter.

[0119] The present invention uses bottom and side laser sensors to obtain laser scanning data of different angles and attributes. The bottom sensor can provide more information about the bottom of the vehicle, while the side sensor can provide side information. This multi-sensor fusion helps to integrate information from different perspectives and improve the accuracy of vehicle identification. The laser power corresponding to the first laser is greater than that of the second laser, which means that the first laser performs reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data during measurement, and obtains reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data with higher brightness and sensitivity, which can improve the measurement performance under low light conditions and ensure data quality. The density data corresponding to the first laser is greater than that of the second laser, which indicates that the first laser can provide denser data points, thereby describing the characteristics of the vehicle in more detail, which helps to improve the accuracy and details of vehicle identification.

[0120] Optionally, the generating laser point cloud data from the laser scanning data to obtain the laser point cloud data includes:

[0121] S121, extracting reflection intensity data from the laser scanning data to obtain reflection intensity data;

[0122] In one embodiment of the present application, reflection intensity information is extracted from the laser scanning data, which may be a reflection intensity value measured by a sensor, and the reflection intensity data of each point is aggregated to obtain the reflection intensity data of the global laser scan.

[0123] In one embodiment of the present application, reflection intensity data is extracted from laser scanning data, and the laser scanning data includes information such as distance, angle, and reflection intensity. The reflection intensity represents the reflection intensity of the laser beam on the surface of the object. After extracting the reflection intensity data, the device obtains the reflection intensity information of each laser point.

[0124] S122, performing reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data, to obtain reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data respectively;

[0125] In one embodiment of the present application, the reflection intensity mean, standard deviation and gradient are calculated to describe the reflection characteristics of each point cloud data point. The reflection intensity mean data represents the average reflection intensity of the entire point cloud. The reflection intensity standard deviation data represents the degree of discreteness of the reflection intensity in the point cloud. The reflection intensity gradient data describes the change of the reflection intensity in the point cloud.

[0126] S123, using a preset reflection intensity material classification model to identify the reflection intensity mean data, the reflection intensity standard deviation data, and the reflection intensity gradient data to obtain reflection intensity material data, wherein the reflection intensity material classification model is constructed and generated by using a support vector machine algorithm using preset historical reflection intensity mean data, historical reflection intensity standard deviation data, historical reflection intensity gradient data, and corresponding material label data;

[0127] In one embodiment of the present application, based on a pre-trained support vector machine model, the mean, standard deviation and gradient data are used to classify the reflection intensity material of each point. The reflection intensity material data is used to describe the reflection characteristics of different parts in the point cloud, such as metal, plastic or glass.

[0128] S124, marking the laser scanning data using the reflection intensity material data corresponding to the reflection intensity data to obtain laser scanning marking data;

[0129] In one embodiment of the present application, the reflection intensity material data is associated with the laser scanning data so as to mark the reflection intensity material of each point in the point cloud.

[0130] S125, generating point cloud data from the laser scanning mark data to obtain laser point cloud data;

[0131] In one embodiment of the present application, a laser point cloud with reflection intensity information is generated based on laser scanning data with reflection intensity marks. The position and reflection intensity information of each point are included in the generated point cloud to obtain complete point cloud data.

[0132] The step of extracting reflection intensity data from the laser scanning data to obtain reflection intensity data includes:

[0133] S1231, performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data;

[0134] In one embodiment of the present application, a small portion of the point cloud in the laser scanning data is selected as a local point cloud, usually using a certain window or region selection method. A point cloud fitting algorithm (e.g., least squares method, Gaussian surface fitting, etc.) is used to perform surface fitting on the selected local point cloud, and a mathematical model representing the local surface is generated, which is usually a curved surface or a plane. The local point cloud surface fitting data is stored as a local point cloud surface fitting data set, which includes the parameters of the fitting model, the coordinates of the point cloud, and other relevant information.

[0135] In one embodiment of the present application, a local area, such as a cylindrical surface, is selected from the laser scanning data, and then the point cloud data of the area is surface fitted using a common point cloud processing algorithm (such as the least squares method). After fitting, a set of local point cloud surface fitting data is obtained, which includes the parameters of the fitted geometric shape, such as the radius, height, and center position of the cylinder.

[0136] S1232, calculating the normal direction of the local point cloud surface fitting data to obtain local point cloud normal direction data;

[0137] In one embodiment of the present application, the normal direction of each point is calculated using the parameters of the fitting model, such as the surface equation or the plane equation. For surface fitting, the gradient of the surface equation can be used to calculate the normal. The normal direction is usually a unit vector, indicating the direction perpendicular to the surface. The calculated normal direction data is associated with each point.

[0138] In one embodiment of the present application, the normal direction of each point is calculated using the local point cloud surface data obtained by fitting. The normal direction represents the direction of the surface at that point, usually perpendicular to the surface. This can be estimated by the neighborhood of the point in the point cloud data. For example, for the surface of a cylinder, the normal direction will be perpendicular to the side of the cylinder.

[0139] S1233, performing curvature calculation on the local point cloud surface fitting data to obtain local point cloud curvature data;

[0140] In one embodiment of the present application, the curvature of each point is calculated using the parameters of the fitting model and the normal direction. The curvature calculation usually involves the second-order derivative of the fitting model or other curvature measurement formulas. The curvature data is usually a scalar value, indicating the degree of curvature of the point cloud surface at each point. The calculated curvature data is associated with each point.

[0141] In one embodiment of the present application, curvature calculation refers to calculating the curvature value of each point, which represents the curvature degree of the surface at that point. The curvature value can be used to determine the flatness or curvature degree of the surface. Curvature calculation is usually completed based on the normal direction and the points in the neighborhood.

[0142] S1234, dividing the laser scanning data according to the local point cloud normal direction data and the local point cloud curvature data to obtain a laser scanning sub-data set;

[0143] In one embodiment of the present application, laser scanning data is partitioned according to local point cloud normal direction data and local point cloud curvature data to generate laser scanning sub-datasets. Data partitioning can be performed according to specific standards or conditions to select point cloud subsets with specific normal directions and curvature attributes.

[0144] In one embodiment of the present application, the laser scanning data is divided into different sub-data sets according to the normal direction and curvature data. For example, determining which points belong to a flat surface and which points belong to a curved surface according to the normal direction will help extract the reflection intensity data in the subsequent steps.

[0145] S1235. Extract reflection intensity data from the laser scanning sub-dataset to obtain reflection intensity data.

[0146] In one embodiment of the present application, performing reflection intensity data extraction on a laser scanning sub-dataset involves extracting reflection intensity values ​​from a selected point cloud subset and associating them with each point.

[0147] In one embodiment of the present application, based on the local point cloud sub-dataset, the reflection intensity data of the points in each sub-dataset is extracted. The reflection intensity data represents the intensity information of the light returning after being emitted from the laser sensor and hitting the target. The intensity value provided by the sensor is used to mark different characteristics of the local area, such as the reflection properties of different materials or the roughness of different surfaces.

[0148] In the present invention, by analyzing the mean, standard deviation and gradient of the reflection intensity data, combined with the preset material classification model, the material can be identified, which helps to determine the structural material of the vehicle surface, such as metal, plastic, rubber, etc., which is very important for further analysis and identification of the vehicle. By fitting the local point cloud surface, calculating the normal direction and curvature, the local feature information of each point can be obtained. These features help to describe the geometric structure of the surface in more detail, including curvature, normal direction, etc. This can help to more accurately identify the surface of different parts of the vehicle. Marking the reflection intensity material data into the laser point cloud data can make each point carry information about its surface material, which can be used for subsequent tasks such as vehicle identification, object detection, etc. Segmenting the laser scanning data according to the local feature data can refine the data into smaller subsets, which helps to understand the characteristics of the vehicle surface in more detail. Using the support vector machine algorithm to construct a reflection intensity material classification model can efficiently identify the material, while improving the robustness and scalability of the system. Through multi-dimensional data fusion and reflection intensity material recognition, it is helpful to more accurately analyze the vehicle surface and improve the performance and reliability of the intelligent wheel axle recognition system.

[0149] Optionally, performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data includes:

[0150] Performing random point sampling and cluster calculation on the laser scanning data to obtain random point sampling data and laser scanning cluster feature data;

[0151] In one embodiment of the present application, a portion of data points are randomly selected from the laser scanning data to reduce the complexity of the point cloud data, and clustering calculations are performed on the randomly sampled data points to identify the local area of ​​the shape or object.

[0152] Screening the random point adoption data and preset threshold distance data to obtain random point screening data;

[0153] In one embodiment of the present application, random point sampling data is filtered based on a preset distance threshold, and only points within a specific distance are retained.

[0154] In one embodiment of the present application, the radius threshold is not fixed in advance, but is adaptively adjusted according to the local characteristics of the point cloud. The specific steps are as follows: First, an initial smaller radius threshold is selected as the starting value; for each point, the number of neighbor points within the radius is calculated; if the number of neighbor points is lower than a preset threshold, it means that the local density of the point cloud is low, and the radius threshold can be increased at this time; the increased radius threshold performs reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data, and obtains reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data respectively, which will cause some points to enter the neighborhood of the point, so the number of neighbor points of these points needs to be recalculated to ensure that the local density meets the preset requirements; repeatedly iterate the above steps until a certain stop condition is met, such as reaching the preset maximum radius threshold or the local density meets specific requirements; finally, the local density information of each point is calculated according to the adaptively adjusted radius threshold, so that the local density changes in different areas in the point cloud can be more accurately reflected, rather than just relying on a fixed threshold; the random point screening data and the preset threshold radius data are density calculated to obtain the first laser scanning density data.

[0155] In one embodiment of the present application, based on a preset radius threshold, the number of points around each point in the random point screening data is calculated to obtain local density information.

[0156] Performing density calculation on the laser scanning cluster feature data to obtain second laser scanning density data;

[0157] In one embodiment of the present application, for each cluster or local area in the laser scanning cluster feature data, the density information of its internal points is calculated.

[0158] In one embodiment of the present application, for each cluster or local area, the density information of its internal points can be calculated. The density calculation can be performed in the following manner: for each cluster, a center point or a reference point is selected. A radius range is defined to represent the local area for calculating the density. The radius can be fixed or adaptively adjusted according to local characteristics. The points within the radius are counted, and these points are considered to be points within the local area. The density is calculated by counting the number of points within the local area. This can be a simple point count or a more complex density function calculation. The calculated density information is associated with other feature data of the cluster to obtain second laser scanning density data.

[0159] Performing multi-scale local neighborhood mapping according to the first laser scanning density data and the second laser scanning density data to obtain first multi-scale local neighborhood data and second multi-scale local neighborhood data, wherein the multi-scale segmentation mapping is generated by mapping through preset scale space pyramid data, and the preset scale space pyramid data is neighborhood radius threshold data corresponding to different laser scanning density threshold data;

[0160] In one embodiment of the present application, based on the first laser scanning density data and the second laser scanning density data, multi-scale local neighborhood mapping is performed using preset scale space pyramid data to capture local structure information of point clouds at different scales.

[0161] In one embodiment of the present application, first laser scanning density data and second laser scanning density data are obtained from the laser scanning data, and these data represent local density information of different regions in the point cloud. The first laser scanning density data and the second laser scanning density data may be density values ​​of each local region.

[0162] In one embodiment of the present application, a scale space pyramid needs to be predefined, and the pyramid includes neighborhood radius threshold data corresponding to different laser scanning density threshold data. These threshold data are used to determine local neighborhoods at different scales.

[0163] In one embodiment of the present application, for each local area or cluster, the first laser scanning density data is used to determine the corresponding scale space pyramid level, which is accomplished by comparing the density data and the threshold data to determine the scale level to which it belongs. According to the determined scale level, the corresponding neighborhood radius threshold data is selected, and the threshold is used to define the radius range of the local neighborhood. Based on the selected neighborhood radius, local neighborhood data is extracted from the point cloud, and these data include points in the local point cloud and their feature information. Repeat the above steps to obtain multi-scale local neighborhood data for each local area. These data capture the local structural information of the point cloud at different scales.

[0164] Extracting local neighborhood data from the laser scanning data according to the first multi-scale local neighborhood data, the second multi-scale local domain data, and the random point screening data to obtain local neighborhood data;

[0165] In one embodiment of the present application, local neighborhood data including coordinate information and other attributes of the points are extracted based on the first multi-scale local neighborhood data, the second multi-scale local neighborhood data and the random point screening data.

[0166] Surface fitting parallel calculation is performed on the local neighborhood data to obtain local point cloud surface fitting data.

[0167] In one embodiment of the present application, the local point cloud is surface fitted using the local neighborhood data to obtain an approximate model of the local point cloud surface. The surface fitting can be based on the least squares method or other mathematical methods to approximate the surface shape of the local point cloud.

[0168] In the present invention, through multi-scale local neighborhood mapping, point cloud data can be analyzed at different scales, which helps to better capture the features and structures in the data. Different scale levels correspond to different features or objects, thereby providing more comprehensive information and more fine-grained data operations to provide accurate and reliable data support. Dividing point cloud data into local neighborhoods helps to extract local features near each point, helps to describe the geometric characteristics of the surface in more detail, and reduces computational complexity. By calculating laser scanning density data, data points can be divided into different density clusters, which helps to separate different structures in the point cloud and better understand the uniformity and sparsity of the data. Through parallel calculation of surface fitting, the fitting surface of each local neighborhood can be obtained, and these fitting surfaces can be used for subsequent analysis and processing, such as feature extraction, object recognition, etc.

[0169] Optionally, downsampling the point cloud denoised data to obtain point cloud downsampled data includes:

[0170] Performing density clustering calculation on the point cloud denoising data to obtain point cloud density clustering data;

[0171] In one embodiment of the present application, a density clustering algorithm (such as DBSCAN or Mean Shift, etc.) is used to cluster the point cloud denoised data to group the points into clusters with similar density.

[0172] Perform density cluster standard deviation calculation and density cluster density calculation on the point cloud density clustering data to obtain a density cluster standard deviation data set and a density cluster density data set;

[0173] In one embodiment of the present application, the standard deviation of the internal points of each density cluster is calculated to measure the dispersion of the points in the cluster. The density of each density cluster, that is, the number of points in the cluster, is calculated to measure the compactness of the cluster.

[0174] Performing density difference processing on the density standard deviation data of the density standard deviation data and the density cluster density data of the density cluster density data set to obtain density difference data, wherein the density difference data includes uniform density data and non-uniform density data;

[0175] In one embodiment of the present application, based on the density cluster standard deviation data and the density cluster density data, it is determined which density clusters are considered to have uniform density (having lower standard deviation and higher density) and which density clusters are considered to have non-uniform density (having higher standard deviation and lower density).

[0176] When it is determined that the density difference data is density uniform data, voxel grid downsampling processing is performed on the point cloud denoising data to obtain the first point cloud downsampling data;

[0177] In one embodiment of the present application, if the density difference data indicates that the point cloud is relatively uniform, a voxel grid downsampling method is used to uniformly select a portion of points in the point cloud to reduce the complexity of the point cloud.

[0178] In one embodiment of the present application, when the density difference data indicates that the point cloud is relatively uniform, a voxel grid downsampling method can be used. This method aims to reduce the complexity of the point cloud while retaining the overall structure of the point cloud. The following are the specific steps: Step 1: Determine that the density difference data indicates that the point cloud is relatively uniform, which is accomplished by comparing the density cluster standard deviation data and the density cluster density data. Step 2: Create a voxel grid according to the required downsampling density (or sampling rate). A voxel is a three-dimensional cube that covers the entire point cloud area. Step 3: Overlay the voxel grid on the point cloud and select a representative point in each voxel, usually the center point of the point in the voxel. Step 4: After selecting the representative point, the required downsampled point cloud data is obtained, that is, the first point cloud downsampling data. These points are representative points in the original point cloud, and the number is relatively small, but the overall structure and features are retained.

[0179] When it is determined that the density difference data is uniform density data, the point cloud denoising data is statistically downsampled according to the density cluster standard deviation data set and the density cluster density data set to obtain the second point cloud downsampled data, wherein the point cloud downsampled data includes the first point cloud downsampled data and the second point cloud downsampled data.

[0180] In one embodiment of the present application, if the density difference data indicates that the point cloud is relatively uneven, a statistical downsampling method can be used to sample the points in each cluster based on the density cluster standard deviation data and the density cluster density data to retain important feature points in the point cloud.

[0181] In one embodiment of the present application, when the density difference data indicates that the point cloud is relatively uneven, a statistical downsampling method can be used. This method focuses more on retaining points with important features rather than uniformly sampled points. The following are the specific steps: Step 1: Determine that the density difference data indicates that the point cloud is relatively uneven, such as analyzing the density cluster standard deviation data and the density cluster density data. Step 2: Statistically downsample each density cluster, such as through the density cluster standard deviation and density cluster density of each density cluster. Step 3: Within each density cluster, a statistical strategy (such as mean, median, etc.) can be used to select representative points. Typically, points that are farther from the center of the density cluster are selected to ensure that points with important features are retained. Step 4: Combine the representative points selected from each density cluster to obtain the second point cloud downsampling data. These points can better retain the important features in the point cloud, rather than just uniformly sampled points.

[0182] In the present invention, by using density clustering calculation, the method can perceive the density distribution in the point cloud data, and judge the density uniformity of the point cloud according to the density cluster standard deviation and density cluster density data, so that the downsampling process is more intelligent, and the downsampling strategy is determined according to the actual density of the point cloud. When the point cloud density is relatively uniform, voxel grid downsampling can effectively retain the main features in the point cloud and reduce the amount of data, which helps to reduce the computational complexity in subsequent processing while maintaining key information. When the point cloud density is uneven, statistical downsampling can be used to capture density changes in more detail, ensuring that more points are retained in areas with higher density, thereby better reflecting the characteristics of the actual scene.

[0183] Optionally, the point cloud adopted data includes first point cloud adopted data and second point cloud adopted data, the first point cloud adopted data and the second point cloud adopted data are point cloud sampling data corresponding to laser adopted data collected from the same vehicle by different sensors or at different angles, and performing point cloud registration on the point cloud adopted data to obtain laser dot cloud data includes:

[0184] Extracting description points from the data used in the first point cloud and the data used in the second point cloud to obtain description point data;

[0185] In one embodiment of the present application, feature extraction algorithms (such as SIFT, ORB, Harris corner detection, etc.) are used to extract feature points from the first point cloud downsampled data and the second point cloud downsampled data. These feature points may be points with unique properties.

[0186] Perform feature point matching on the data used in the first point cloud and the data used in the second point cloud according to the description point data to obtain feature point matching data;

[0187] In one embodiment of the present application, a feature matching algorithm (such as RANSAC, ICP, matching based on feature descriptors, etc.) is used to match the feature points in the first point cloud downsampled data and the second point cloud downsampled data to find the corresponding relationship between them. SIFT is a feature detection and description algorithm based on scale and rotation invariance, which can detect the same key points at different scales and angles, so it is very effective in processing image matching under different viewing angles and lighting conditions. Scale space extreme value detection, by applying Gaussian filters at different scales, detects extreme points in the image, which usually correspond to features with different scales. Key point positioning, by comparing the gradients of pixels around the extreme points to determine the exact position of the key points. ORB is a feature detection and description algorithm with fast calculation speed and rotation invariance, which is very useful in real-time applications, such as SLAM (Simultaneous Localization and Mapping). FAST corner detection: Use the FAST (Features from Accelerated Segment Test) algorithm to detect key points in the image. Direction assignment: Assign a main direction to each key point to enhance rotation invariance. Harris corner detection is a classic corner detection method, which is used to detect corners in an image. These corners usually correspond to corners and edges in the image. Gradient calculation, calculate the gradient of each pixel in the image. Structure tensor calculation, calculate the structure tensor of each pixel based on the gradient information, including the Harris matrix. Corner response calculation, calculate the corner response function value of each pixel based on the structure tensor. Non-maximum suppression, select pixels with high corner response values ​​in the image as key points.

[0188] generating preliminary transformation data according to the feature point matching data;

[0189] In one embodiment of the present application, a preliminary point cloud transformation, such as a rigid transformation (translation, rotation, scaling, etc.), is calculated using matched feature point data to attempt to align the first point cloud downsampled data to the second point cloud downsampled data.

[0190] Using the preliminary transformed data, performing least squares iterative optimization on the data used in the first point cloud and the data used in the second point cloud to obtain optimized transformed data;

[0191] In one embodiment of the present application, the point cloud is iteratively optimized based on the preliminary transformation data using the least squares method or other optimization algorithms to obtain more accurate transformation parameters, thereby better aligning the first point cloud downsampled data and the second point cloud downsampled data.

[0192] Point cloud fusion is performed on the data used in the first point cloud and the data used in the second point cloud according to the optimized transformation data to obtain laser point cloud data.

[0193] In one embodiment of the present application, the first point cloud downsampled data is fused into the second point cloud downsampled data using optimized transformation parameters, or they are fused into a common coordinate system to obtain laser point cloud data, which will merge the two point clouds to obtain more complete information.

[0194] In the present invention, by simultaneously using the first point cloud down-sampled data and the second point cloud down-sampled data collected by different sensors, the method allows for more comprehensive capture of vehicle or scene information, and this multi-sensor data fusion helps to improve the accuracy and completeness of the data. Using description point extraction and feature point matching, the correspondence between the first point cloud down-sampled data and the second point cloud down-sampled data can be determined. By applying the least squares iterative optimization, the alignment quality of the point cloud can be further improved, ensuring that the point cloud data collected by different sensors or at different angles maintain an accurate spatial relationship after fusion. The method fuses the first point cloud down-sampled data and the second point cloud down-sampled data into a unified laser dot cloud data, which is helpful for subsequent processing and analysis, such as intelligent wheel axle recognition.

[0195] Optionally, the vehicle feature data includes vehicle color feature data, vehicle texture feature data, vehicle shape feature data and vehicle local feature data, and the feature extraction of the vehicle image data to obtain the vehicle feature data includes:

[0196] Extracting vehicle color features from the vehicle image data to obtain vehicle color feature data;

[0197] In one embodiment of the present application, image processing techniques, such as color histogram, color channel statistics, color space transformation, etc., are used to extract color feature information from vehicle images, including extraction of main colors, color distribution, color histogram, etc.

[0198] Extracting vehicle texture features from the vehicle image data to obtain the vehicle texture feature data;

[0199] In one embodiment of the present application, texture analysis methods, such as local binary pattern (LBP), histogram of oriented gradients (HOG), texture filters, etc., are used to extract texture feature information from vehicle images, including texture direction, texture density, texture histogram, etc.

[0200] Extracting vehicle shape features from the vehicle image data to obtain vehicle shape feature data;

[0201] In one embodiment of the present application, shape analysis techniques, such as edge detection, contour extraction, shape descriptors, etc., are used to extract shape feature information from vehicle images, including the outer contour, shape parameters, shape descriptors, etc. of the vehicle.

[0202] The vehicle local feature extraction is performed on the vehicle image data to obtain the vehicle local feature data.

[0203] In one embodiment of the present application, local feature detection and description methods, such as key point detection, feature point descriptor extraction (such as SIFT, SURF, etc.), local area division, etc., are used to extract local feature information from the vehicle image, including local key points and local feature descriptors of the vehicle.

[0204] By extracting the vehicle color, texture, shape and local features in the present invention, the method can more comprehensively describe the characteristics of the vehicle to be measured, which helps to improve the accuracy of intelligent wheel axle recognition because it includes not only point cloud data but also visual information. Fusion of different types of vehicle feature data can provide more representative feature vectors, which helps to distinguish different vehicles. Covering multiple feature types makes the method applicable to vehicles of different types and styles, and the vehicle can be identified by this method regardless of its color, texture, shape or local features.

[0205] Optionally, performing multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fused vehicle data includes:

[0206] Performing data alignment on the laser dot cloud data and the vehicle feature data to obtain dot vehicle feature alignment data;

[0207] In one embodiment of the present application, for the laser dot cloud data and the vehicle feature data, it is ensured that they have consistent representation in the same coordinate system or feature descriptor, including associating the position of the vehicle feature data with the laser dot cloud data so that they correspond to the same spatial coordinates.

[0208] Performing self-attention calculation on the dot matrix vehicle feature alignment data to obtain aligned self-attention data;

[0209] In one embodiment of the present application, a self-attention neural network or an attention mechanism is used to calculate weights of features in the dot matrix vehicle feature alignment data to capture the correlation and importance between different features and help identify the association information between the laser dot matrix cloud data and vehicle features.

[0210] The aligned self-attention data is subjected to feature weighted fusion processing to obtain vehicle multi-dimensional fusion data; or, the aligned self-attention data is subjected to feature connection to obtain vehicle multi-dimensional fusion data.

[0211] In one embodiment of the present application, based on the weight information of the aligned self-attention data, feature weighted fusion can be performed, that is, laser dot cloud data and vehicle feature data are weightedly combined to generate fused multi-dimensional data. Another method is feature connection, that is, connecting the feature vectors of the two together to form a fused feature vector with a higher dimension.

[0212] In the present invention, the laser dot cloud data and the vehicle feature data are fused to fully integrate the information from different sensor sources. This helps to improve the accuracy of vehicle recognition because different data sources provide different perspectives and features. Through self-attention calculation, the system can better focus on the features related to the recognition task and automatically learn the importance between different features, which helps to reduce the impact of irrelevant or noise features and improve the stability of recognition. In the process of multi-dimensional data fusion, the self-attention data is aligned for feature weighted fusion, and each feature can be weighted according to its importance, thereby improving the overall fusion effect. By fusing multiple feature data and automatically learning feature importance, this method can improve the accuracy of vehicle recognition and help to more accurately identify and track target vehicles.

[0213] Optionally, the present application further provides an intelligent wheel axle recognition device based on laser + visual AI technology, which is used to execute the intelligent wheel axle recognition method based on laser + visual AI technology as described above, and the intelligent wheel axle recognition device based on laser + visual AI technology includes:

[0214] The laser scanning module is used to scan the vehicle to be measured through a laser sensor to obtain laser dot cloud data;

[0215] The vehicle image acquisition module is used to acquire vehicle images through a high-definition camera to obtain vehicle image data;

[0216] A feature extraction module, used to extract features from the vehicle image data to obtain vehicle feature data;

[0217] The multi-dimensional data fusion module is used to perform multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle for performing intelligent wheel axle recognition operations.

[0218] In one embodiment of the present application, a new type of non-contact intelligent axle identifier based on laser + visual AI technology is provided. The device is composed of a mounting column, a laser sensor, a high-definition camera, an edge computing unit and other parts. It adopts an integrated design and is installed sideways, which is convenient for deployment and maintenance. The laser sensor is used to scan and measure the vehicle to form laser dot cloud data; the vehicle image is collected by a high-definition camera to extract vehicle feature data; the multi-dimensional data is fused using the laser vision fitting algorithm deployed in the edge computing unit to achieve accurate identification of the number of wheels, the number of axles, and the vehicle model in non-stop scenarios. Laser plus video fusion enables the device to have the characteristics of high measurement accuracy, strong anti-interference ability, and strong environmental adaptability. It can be widely used in various scenarios such as highway entrances and exits, non-stop inspection stations, and freight source companies.

[0219] The purpose of the present invention is that the method first obtains laser dot cloud data through a laser sensor, which can provide accurate information about the external structure and shape of the vehicle. At the same time, the vehicle image data is obtained through a high-definition camera, which can capture the color, texture, logo and other visual information of the vehicle. The two different data sources are fused to achieve comprehensive utilization of information, which is helpful for a more comprehensive understanding and identification of the vehicle. The laser dot cloud data provides high-resolution vehicle external information, including the geometric features of the axle. By extracting features from these data, the position and shape of the axle of the vehicle can be accurately identified, providing accurate and reliable data support for the accurate positioning and identification of the axle, especially in traffic monitoring, intelligent transportation systems and vehicle management. The vehicle image data provides visual information such as color, texture and logo, which helps to further verify the accuracy of the laser data, as well as the identification and classification of the vehicle. For example, the color and logo of the vehicle can be used to distinguish different types of vehicles, such as trucks, cars or buses. In step S4, the method merges the laser dot cloud data and the vehicle image data into a unified data set through multidimensional data fusion. Multidimensional data fusion helps to integrate data from different sources into a comprehensive representation, improving the comprehensive understanding and identification accuracy of the vehicle axle. The data generated by the present invention can realize intelligent wheel axle identification operation. Through this method, accurate and reliable non-contact data support can be provided for the automatic detection and identification of vehicle wheel axles, and it has broad application prospects in the fields of traffic monitoring, intelligent transportation systems, road safety and vehicle management. The system can intelligently identify the position, number and shape of wheel axles based on the comprehensive information of laser and visual data, providing a more powerful tool for traffic management and monitoring.

[0220] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0221] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. An intelligent wheel axle recognition method based on laser + visual AI technology, characterized in that: The method comprises: S1, scanning the vehicle to be measured by a laser sensor to obtain laser dot cloud data; S2, collecting vehicle images through a high-definition camera to obtain vehicle image data; S3, extracting features from the vehicle image data to obtain vehicle feature data; S4, performing multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle to perform intelligent wheel axle recognition operation; The laser sensor is used to scan the vehicle to be measured to obtain laser dot cloud data, including: Control the laser sensor to emit laser light to scan the vehicle to be measured and receive reflected light to obtain laser scanning data; Generating laser point cloud data from the laser scanning data to obtain laser point cloud data; Performing noise reduction processing on the laser point cloud data to obtain point cloud noise reduction data; Downsampling the point cloud denoised data to obtain point cloud downsampled data; Performing point cloud registration on the point cloud down-sampling data to obtain laser point cloud data; The step of generating laser point cloud data from the laser scanning data to obtain the laser point cloud data comprises: Extracting reflection intensity data from the laser scanning data to obtain reflection intensity data; Performing reflection intensity mean calculation, reflection intensity standard deviation calculation and reflection intensity gradient calculation on the reflection intensity data to obtain reflection intensity mean data, reflection intensity standard deviation data and reflection intensity gradient data respectively; The reflection intensity mean data, the reflection intensity standard deviation data and the reflection intensity gradient data are identified by using a preset reflection intensity material classification model to obtain reflection intensity material data, wherein the reflection intensity material classification model is constructed and generated by using a support vector machine algorithm using preset historical reflection intensity mean data, historical reflection intensity standard deviation data, historical reflection intensity gradient data and corresponding material label data; Marking the laser scanning data using the reflection intensity material data corresponding to the reflection intensity data to obtain laser scanning marking data; Generating point cloud data from the laser scanning mark data to obtain laser point cloud data; The step of extracting reflection intensity data from the laser scanning data to obtain reflection intensity data includes: Performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data; Calculating the normal direction of the local point cloud surface fitting data to obtain local point cloud normal direction data; Performing curvature calculation on the local point cloud surface fitting data to obtain local point cloud curvature data; Dividing the laser scanning data according to the local point cloud normal direction data and the local point cloud curvature data to obtain a laser scanning sub-data set; Reflection intensity data is extracted from the laser scanning sub-dataset to obtain reflection intensity data.

2. The method according to claim 1, characterized in that The laser scanning data includes first laser scanning data and second laser scanning data. The laser sensor is controlled to emit laser to scan the vehicle to be measured and receive reflected light to obtain laser scanning data, including: Controlling the laser sensor installed at the bottom to emit a first laser to scan the vehicle to be measured and receive a first reflected light to obtain the first laser scanning data; The laser sensor installed on the side is controlled to emit a second laser to scan the vehicle to be measured and receive the second reflected light to obtain the second laser scanning data, wherein the scanning angle of the first laser is smaller than the scanning angle of the second laser, the laser power corresponding to the first laser is larger than the laser power corresponding to the second laser, and the sampling density data corresponding to the first laser is larger than the sampling density data of the second laser.

3. The method according to claim 1, characterized in that The performing local point cloud surface fitting on the laser scanning data to obtain local point cloud surface fitting data includes: Performing random point sampling and cluster calculation on the laser scanning data to obtain random point sampling data and laser scanning cluster feature data; Screening the random point sampling data and the preset threshold distance data to obtain random point screening data; Performing density calculation on the random point screening data and the preset threshold radius data to obtain first laser scanning density data; Performing density calculation on the laser scanning cluster feature data to obtain second laser scanning density data; Performing multi-scale local neighborhood mapping according to the first laser scanning density data and the second laser scanning density data to obtain first multi-scale local neighborhood data and second multi-scale local neighborhood data; Extracting local neighborhood data from the laser scanning data according to the first multi-scale local neighborhood data, the second multi-scale local neighborhood data, and the random point screening data to obtain local neighborhood data; Performing surface fitting parallel calculation on the local neighborhood data to obtain local point cloud surface fitting data.

4. The method according to claim 1, characterized in that: The step of downsampling the point cloud denoised data to obtain point cloud downsampled data includes: Performing density clustering calculation on the point cloud denoising data to obtain point cloud density clustering data; Perform density cluster standard deviation calculation and density cluster density calculation on point cloud density clustering data to obtain density cluster standard deviation data set and density cluster density data set; Performing density difference processing on the density cluster standard deviation data of the density cluster standard deviation data set and the density cluster density data of the density cluster density data set to obtain density difference data, wherein the density difference data includes uniform density data and non-uniform density data; When it is determined that the density difference data is uniform density data, voxel grid downsampling processing is performed on the point cloud denoising data to obtain first point cloud downsampling data; When it is determined that the density difference data is uniform density data, the point cloud denoising data is statistically downsampled according to the density cluster standard deviation data set and the density cluster density data set to obtain second point cloud downsampled data, wherein the point cloud downsampled data includes first point cloud downsampled data and second point cloud downsampled data.

5. The method according to claim 1, characterized in that The point cloud down-sampling data includes first point cloud down-sampling data and second point cloud down-sampling data, the first point cloud down-sampling data and the second point cloud down-sampling data are point cloud down-sampling data corresponding to laser sampling data collected from the same vehicle by different sensors or at different angles, and the point cloud down-sampling data is point cloud registered to obtain laser dot cloud data, including: Extracting description points from the first point cloud downsampled data and the second point cloud downsampled data to obtain description point data; Performing feature point matching on the first point cloud downsampled data and the second point cloud downsampled data according to the description point data to obtain feature point matching data; generating preliminary transformation data according to the feature point matching data; Using the preliminary transformation data, performing least squares iterative optimization on the first point cloud downsampled data and the second point cloud downsampled data to obtain optimized transformation data; Point cloud fusion is performed on the first point cloud down-sampling data and the second point cloud down-sampling data according to the optimized transformation data to obtain laser point cloud data.

6. The method according to claim 1, characterized in that The vehicle feature data includes vehicle color feature data, vehicle texture feature data, vehicle shape feature data and vehicle local feature data, and the feature extraction of the vehicle image data to obtain the vehicle feature data includes: Extracting vehicle color features from the vehicle image data to obtain vehicle color feature data; Extracting vehicle texture features from the vehicle image data to obtain the vehicle texture feature data; Extracting vehicle shape features from the vehicle image data to obtain vehicle shape feature data; The vehicle local feature extraction is performed on the vehicle image data to obtain the vehicle local feature data.

7. The method according to claim 1, characterized in that The multi-dimensional data fusion of the laser dot cloud data and the vehicle feature data to obtain the vehicle multi-dimensional fusion data includes: Performing data alignment on the laser dot cloud data and the vehicle feature data to obtain dot vehicle feature alignment data; Performing self-attention calculation on the dot matrix vehicle feature alignment data to obtain aligned self-attention data; The aligned self-attention data is subjected to feature weighted fusion processing to obtain vehicle multi-dimensional fusion data; or, the aligned self-attention data is subjected to feature connection to obtain vehicle multi-dimensional fusion data.

8. An intelligent wheel axle recognition device based on laser + visual AI technology, characterized in that: Used to execute the intelligent wheel axle recognition method based on laser + visual AI technology as claimed in claim 1, the intelligent wheel axle recognition device based on laser + visual AI technology comprises: The laser scanning module is used to scan the vehicle to be measured through a laser sensor to obtain laser dot cloud data; The vehicle image acquisition module is used to acquire vehicle images through a high-definition camera to obtain vehicle image data; A feature extraction module, used to extract features from the vehicle image data to obtain vehicle feature data; The multi-dimensional data fusion module is used to perform multi-dimensional data fusion on the laser dot cloud data and the vehicle feature data to obtain multi-dimensional fusion data of the vehicle for performing intelligent wheel axle recognition operations.

Images