Image-based three-dimensional point cloud traffic sign classification method and system

Abstract

The application provides a kind of three-dimensional point cloud traffic sign classification method and system based on image, it is related to computer vision technical field, including: the multiple frames of multiple angle images of traffic sign are collected and converted into point cloud data, the optimal segmentation threshold is determined based on the histogram constructed by reflection intensity and geometric feature, the spatial distribution characteristics of scattering coefficient and absorption coefficient are combined to carry out region merging, obtain the point cloud sub-region of uniform material, the core area is determined using curvature entropy value and normal vector entropy value and local coordinate system is established, region growing and dynamic merging are carried out through spatial position and geometric structure constraint, obtain candidate point cloud sub-region, extract position offset and attitude change to construct time sequence motion feature, realize the accurate classification of traffic sign.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image-based three-dimensional point cloud traffic sign classification method and system. Background Art

[0002] With the rapid development of intelligent transportation systems and autonomous driving technologies, accurate identification and classification of traffic signs have become key to ensuring traffic safety and improving traffic efficiency. Image-based 3D point cloud technology, which can provide rich spatial information and geometric features, has shown significant advantages in the field of traffic sign recognition. Currently, existing technologies mainly rely on 3D point cloud data obtained by lidar and visual sensors to achieve traffic sign classification through feature extraction and pattern recognition methods.

[0003] However, existing technologies still ignore the impact of material features on classification when dealing with traffic sign classification, relying solely on geometric features for region segmentation. This makes it difficult to accurately distinguish different types of traffic signs with similar geometric shapes in complex environments. There is a lack of temporal correlation analysis of target motion states in continuous multi-frame data, making it difficult to effectively handle changes in sign posture caused by vehicle movement or environmental changes. Furthermore, the importance differences of local regions are not fully considered during feature extraction, and uniform feature weights are used, resulting in limited classification accuracy. This is particularly problematic when dealing with partial occlusion or changing lighting conditions.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for classifying three-dimensional point cloud traffic signs based on images, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for classifying traffic signs based on a three-dimensional point cloud image, comprising:

[0007] Collect continuous multi-frame and multi-angle image data of traffic signs to be classified, and obtain initial point cloud data based on camera parameters;

[0008] Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing a reflection intensity histogram and a geometric feature histogram and determining an optimal segmentation threshold, dividing the initial point cloud data into multiple initial sub-regions according to the optimal segmentation threshold, calculating the scattering coefficient and absorption coefficient of each initial sub-region, and merging adjacent regions based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material;

[0009] Calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select the regions that meet the pre-set constraints and dynamically merge them to obtain candidate point cloud sub-regions;

[0010] The position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data are extracted, the motion state descriptor is constructed and time series association is performed, the target feature vector is obtained and input into the classifier to obtain the traffic sign classification result.

[0011] In an optional embodiment,

[0012] Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing the reflection intensity histogram and geometric feature histogram, and determining the optimal segmentation threshold includes:

[0013] Acquire initial point cloud data comprising a plurality of point cloud data points, extract the material reflection coefficient, atmospheric attenuation coefficient, laser propagation distance, and incident angle of each point cloud data point in the initial point cloud data, and calculate a reflection intensity value;

[0014] Constructing a local neighborhood point set for each point cloud data point, extracting eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating local curvature values ​​based on the eigenvalues, obtaining normal vectors between the point cloud data point and each point in the local neighborhood point set, and calculating consistency values ​​of the normal vectors;

[0015] Counting the distribution characteristics of the reflection intensity values ​​to construct a reflection intensity histogram, counting the distribution characteristics of the local curvature values ​​and the consistency values ​​of the normal vectors to construct a geometric feature histogram, and calculating the normalized probability distribution of the reflection intensity histogram and the geometric feature histogram;

[0016] Based on the normalized probability distribution, the inter-class variance is calculated, and the optimal threshold of reflection intensity and the optimal threshold of geometric features are obtained by maximizing the inter-class variance; the information entropy features of the reflection intensity histogram and the geometric feature histogram are extracted to calculate the feature fusion weight, and the feature fusion weight is applied to the optimal threshold of reflection intensity and the optimal threshold of geometric features respectively to obtain the optimal segmentation threshold.

[0017] In an optional embodiment,

[0018] The initial point cloud data is divided into multiple initial sub-regions according to the optimal segmentation threshold, the scattering coefficient and absorption coefficient of each initial sub-region are calculated, and adjacent regions are merged based on spatial distribution characteristics. The obtained point cloud sub-regions with uniform material include:

[0019] Segmenting the initial point cloud data into a plurality of sub-regions based on the optimal segmentation threshold to obtain a plurality of initial sub-regions;

[0020] collecting scattered light intensity, incident light intensity, and transmitted light intensity of the initial sub-area, calculating a scattering coefficient based on a ratio of the scattered light intensity to the incident light intensity, and calculating an absorption coefficient based on a ratio of the transmitted light intensity to the incident light intensity;

[0021] Extracting the number of point cloud data points in the initial subregion, measuring the bounding volume of the initial subregion, calculating the ratio of the number of point cloud data points to the bounding volume to obtain a spatial distribution characteristic, obtaining the spatial distance between any two point cloud data points in the initial subregion, substituting the spatial distance into a Gaussian function to calculate a spatial continuity coefficient, extracting the scattering coefficient and absorption coefficient of adjacent initial subregions to calculate an optical characteristic distance, and calculating the difference between the spatial distribution characteristics and the spatial continuity coefficient of the adjacent initial subregions to obtain a spatial distribution distance;

[0022] Weight coefficients are applied to the optical characteristic distance and the spatial distribution distance respectively and the sum is calculated to obtain a region merging criterion function value. When the region merging criterion function value is less than a preset threshold, the adjacent initial sub-regions are merged to obtain a point cloud sub-region with uniform material. The merging operation is repeated until the region merging criterion function values ​​of all adjacent point cloud sub-regions are greater than the preset threshold.

[0023] In an optional embodiment,

[0024] Calculating the local curvature entropy and normal vector entropy of the point cloud sub-regions, ranking the importance using the entropy values ​​as weight coefficients, determining the core region, and establishing a local coordinate system based on the core region includes:

[0025] Select a target point in the point cloud sub-region, construct a spherical search region with a preset radius with the target point as the center, extract neighborhood points in the spherical search region to form a local neighborhood point set, and calculate the covariance matrix of the local neighborhood point set;

[0026] Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​arranged in descending order, taking the ratio of the minimum eigenvalue to the sum of the eigenvalues ​​as the local curvature of the target point, traversing all point cloud data points in the point cloud sub-region to repeatedly calculate the local curvature to obtain a local curvature set, and statistically calculating the distribution probability of the local curvature set within a preset interval to obtain a curvature entropy value;

[0027] Mapping the normal vectors of the point cloud data points in the point cloud sub-region to a unit sphere and dividing the sphere into a plurality of direction intervals, and calculating the normal vector distribution probability in each direction interval to obtain a normal vector entropy value;

[0028] Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value, dividing the comprehensive entropy value by the maximum comprehensive entropy value of all point cloud sub-regions to obtain an importance coefficient, calculating the volume of the point cloud sub-region, and taking the product of the importance coefficient and the volume as the region sorting criterion value, and selecting the point cloud sub-region with the maximum region sorting criterion value as the core region;

[0029] The covariance matrix of the core area is calculated and the eigenvector is extracted to determine the direction of the coordinate axis. The center of mass position of the core area is calculated and the coordinate origin is obtained by offsetting the eigenvector direction corresponding to the maximum eigenvalue by a preset distance to establish a local coordinate system.

[0030] In an optional embodiment,

[0031] Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value includes:

[0032] Constructing a local connection graph for the local neighborhood point set and calculating the geodesic distance between any two neighborhood points, projecting the local neighborhood point set into a manifold space, and calculating the mean curvature and Gaussian curvature based on the geodesic distance;

[0033] Calculating a first principal curvature and a second principal curvature based on the mean curvature and the Gaussian curvature, taking a negative exponential of a product of an absolute value of the Gaussian curvature and a preset attenuation factor, and multiplying the product by the difference between the first principal curvature and the second principal curvature plus one to obtain a dynamic manifold weight;

[0034] Extracting the eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating the curvature change rate along the principal curvature direction based on the eigenvalues ​​and the first principal curvature and the second principal curvature to obtain a geometric structure eigenvalue, calculating the geometric structure eigenvalue angle and spatial distance of adjacent local neighborhood points, multiplying the cosine value of the angle by the exponential decay value of the spatial distance to construct a persistence matrix, and calculating the ratio of the maximum eigenvalue of the persistence matrix to the matrix trace to obtain a geometric persistence coefficient;

[0035] Calculate the variance of the geometric structure characteristic value, multiply the preset compensation coefficient by the inverse of the geometric persistence coefficient plus one, and multiply the variance plus the logarithm of one to obtain a dynamic entropy compensation value, multiply the dynamic manifold weight by the weighted sum of the curvature entropy value and the normal vector entropy value, and superimpose the dynamic entropy compensation value to obtain a comprehensive entropy value.

[0036] In an optional embodiment,

[0037] The region growth range is determined by combining the spatial position relationship and the geometric structure relationship, and the regions that meet the pre-set constraints are selected for dynamic merging. The candidate point cloud sub-regions include:

[0038] For each point in the point cloud sub-region, the minimum Euclidean distance between the current point and the core region is calculated to obtain a spatial position relationship, and the dot product of the normal vector of the current point and the average normal vector of the core region is calculated to obtain a geometric structure relationship;

[0039] Performing a product operation on the spatial position relationship and the complement of the geometric structure relationship to obtain a growth constraint value, and determining an area in the point cloud sub-area where the growth constraint value is less than a preset growth threshold as a regional growth range;

[0040] Calculating the shortest distance between adjacent regions within the region growth range to obtain an inter-region distance threshold, calculating the dot product of the normal vectors of the adjacent regions to obtain normal vector consistency, calculating the curvature difference of the adjacent regions to obtain curvature continuity, and performing a weighted combination of the inter-region distance threshold, the normal vector consistency, and the curvature continuity to obtain region similarity;

[0041] Adjacent regions whose region similarity is greater than a preset merging threshold are dynamically merged, the boundary features and region feature parameters of the merged region are updated, and the merging operation is repeated until the region similarity is less than the preset merging threshold to obtain a candidate point cloud sub-region.

[0042] In an optional embodiment,

[0043] Extract the position offset and attitude change of the candidate point cloud sub-region in adjacent multi-frame data, construct the motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier, and obtain the traffic sign classification results including:

[0044] Obtain the spatial coordinates of the candidate point cloud subregion in adjacent multi-frame point cloud data, calculate the coordinate difference between adjacent frames to obtain a position offset matrix, use the coordinate axis direction of the local coordinate system as the main axis direction, calculate the Euler angle change of the candidate point cloud subregion between adjacent frames to obtain a posture change matrix;

[0045] The projection component of the position offset matrix in three-dimensional space and the pitch angle, roll angle, and yaw angle changes of the attitude change matrix are combined into a six-dimensional vector, a motion state descriptor based on a sliding time window is constructed, principal component analysis is performed on the temporal change characteristics of the motion state descriptor to extract the change pattern of the key dimension, and the change patterns in multiple time windows are cascaded to obtain a target feature vector;

[0046] The target feature vector is input into a pre-trained support vector classifier to obtain a traffic sign classification result.

[0047] A second aspect of an embodiment of the present invention provides an image-based three-dimensional point cloud traffic sign classification system, comprising:

[0048] The first unit is used to collect continuous multi-frame multi-angle image data of the traffic sign to be classified and obtain initial point cloud data based on camera parameters;

[0049] The second unit is configured to calculate the reflection intensity value and local geometric characteristic value of each point in the initial point cloud data, construct a reflection intensity histogram and a geometric characteristic histogram, and determine an optimal segmentation threshold. The initial point cloud data is divided into a plurality of initial sub-regions according to the optimal segmentation threshold, and the scattering coefficient and absorption coefficient of each initial sub-region are calculated. Adjacent regions are merged based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material.

[0050] The third unit is used to calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select and dynamically merge the regions that meet the pre-set constraints to obtain candidate point cloud sub-regions;

[0051] The fourth unit is used to extract the position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data, construct a motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier to obtain the traffic sign classification result.

[0052] According to a third aspect of the embodiments of the present invention,

[0053] An electronic device is provided, comprising:

[0054] processor;

[0055] a memory for storing processor-executable instructions;

[0056] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0057] According to a fourth aspect of the embodiments of the present invention,

[0058] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0059] In the present invention, through the collection of multi-angle image data and the processing of point cloud data, the characteristics of traffic signs can be captured more comprehensively. Through the analysis of reflection intensity and geometric characteristics, the segmentation and merging process of point cloud data is optimized, so that the final point cloud sub-area is more uniform in material, which is helpful for subsequent classification work. Combined with motion state descriptors and time series associations, the classifier's ability to recognize dynamic traffic signs is enhanced, and it can adapt to changes in traffic signs under different environments and conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the process of a method for classifying three-dimensional point cloud traffic signs based on images according to an embodiment of the present invention;

[0061] Figure 2 A spatial distribution feature map of point cloud data of the image-based three-dimensional point cloud traffic sign classification method according to an embodiment of the present invention;

[0062] Figure 3 This is a comparison chart of the principal curvature calculation accuracy of the image-based three-dimensional point cloud traffic sign classification method according to an embodiment of the present invention;

[0063] Figure 4 This is a spatial distribution feature map of point cloud data of the image-based three-dimensional point cloud traffic sign classification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0065] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0066] Figure 1 FIG. 1 is a flow chart of a method for classifying three-dimensional point cloud traffic signs based on images according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0067] Collect continuous multi-frame and multi-angle image data of traffic signs to be classified, and obtain initial point cloud data based on camera parameters;

[0068] Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing a reflection intensity histogram and a geometric feature histogram and determining an optimal segmentation threshold, dividing the initial point cloud data into multiple initial sub-regions according to the optimal segmentation threshold, calculating the scattering coefficient and absorption coefficient of each initial sub-region, and merging adjacent regions based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material;

[0069] Calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select the regions that meet the pre-set constraints and dynamically merge them to obtain candidate point cloud sub-regions;

[0070] The position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data are extracted, the motion state descriptor is constructed and time series association is performed, the target feature vector is obtained and input into the classifier to obtain the traffic sign classification result.

[0071] In an optional embodiment,

[0072] Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing the reflection intensity histogram and geometric feature histogram, and determining the optimal segmentation threshold includes:

[0073] Acquire initial point cloud data comprising a plurality of point cloud data points, extract the material reflection coefficient, atmospheric attenuation coefficient, laser propagation distance, and incident angle of each point cloud data point in the initial point cloud data, and calculate a reflection intensity value;

[0074] Constructing a local neighborhood point set for each point cloud data point, extracting eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating local curvature values ​​based on the eigenvalues, obtaining normal vectors between the point cloud data point and each point in the local neighborhood point set, and calculating consistency values ​​of the normal vectors;

[0075] Counting the distribution characteristics of the reflection intensity values ​​to construct a reflection intensity histogram, counting the distribution characteristics of the local curvature values ​​and the consistency values ​​of the normal vectors to construct a geometric feature histogram, and calculating the normalized probability distribution of the reflection intensity histogram and the geometric feature histogram;

[0076] Based on the normalized probability distribution, the inter-class variance is calculated, and the optimal threshold of reflection intensity and the optimal threshold of geometric features are obtained by maximizing the inter-class variance; the information entropy features of the reflection intensity histogram and the geometric feature histogram are extracted to calculate the feature fusion weight, and the feature fusion weight is applied to the optimal threshold of reflection intensity and the optimal threshold of geometric features respectively to obtain the optimal segmentation threshold.

[0077] After acquiring the initial point cloud data, perform data quality assessment and preprocessing. By calculating the point cloud density distribution histogram, determine the density threshold and remove outliers in low-density areas. Spatial position normalization is performed on the point cloud data, mapping the coordinates to a standard space. A statistical outlier filtering method is used to calculate the average distance from each point to its neighboring points. A threshold of multiples of the standard deviation is set to filter out outliers to ensure data quality.

[0078] During the point cloud parameter extraction phase, the material reflectance characteristics of the target surface are retrieved based on a pre-established material spectrum database. The material reflectance coefficient is calculated using a spectral matching algorithm, while also considering the influence of the material's bidirectional reflectance distribution function on the reflectance characteristics. Environmental parameters such as atmospheric visibility, temperature, and humidity are collected in real time to establish an atmospheric attenuation model. A piecewise linear interpolation method is used to calculate the atmospheric attenuation coefficient at different distances. The laser propagation time is calculated using the timestamps of the laser radar's transmitted and received signals, and the precise propagation distance is obtained by combining it with the speed of light. Based on the laser radar's installation position and attitude parameters, the laser beam's emission direction vector is calculated, and the angle of incidence is determined through vector operations combined with the target surface normal vector. The extracted parameters are substituted into the improved laser radar equation, and an iterative optimization method is used to solve the reflection intensity value. To eliminate the influence of distance attenuation, the reflection intensity is compensated and normalized based on a distance-based exponential decay model.

[0079] The geometric feature extraction process determines the local neighborhood range. An adaptive point cloud density estimation method is used to calculate the optimal neighborhood radius based on the local point density distribution. The KD tree is used to accelerate the neighborhood search and obtain the point set within the spherical neighborhood. Principal component analysis is performed on the neighborhood point set to construct the covariance matrix and solve the eigenvalues ​​and eigenvectors. The geometric type of the local area is judged according to the ratio of the eigenvalues, including plane, line and scattered point types. A multi-scale curvature estimation method is used to calculate the curvature value at different neighborhood radii, and the local curvature value is obtained by weighted combination. The local surface normal vector is obtained by least squares plane fitting, and the cosine of the angle between the normal vector of the center point and the normal vector of the neighborhood point is calculated. The normal vector consistency value is calculated using Gaussian weighting, and the weight coefficient decays exponentially with the increase of the point distance.

[0080] During the histogram construction phase, data analysis is performed on the reflection intensity values. The mean, variance, and skewness of the reflection intensity are calculated, and the optimal number of bins is determined based on the data distribution characteristics. An adaptive binning strategy combining equal-frequency binning and equal-width binning is adopted, with smaller binning intervals used in data-dense areas and larger binning intervals used in data-sparse areas. For local curvature values ​​and normal vector consistency values, their numerical distribution patterns are analyzed, and non-uniform binning intervals are designed. More refined divisions are used in intervals with drastic feature changes. After constructing the histogram, a Gaussian kernel function is used to smooth the histogram, and the standard deviation of the kernel function is adaptively adjusted according to the data noise level. The smoothed histogram is normalized to obtain a probability distribution.

[0081] During the optimal threshold solution phase, multi-scale peak detection is performed on the normalized probability distribution. The distribution curve is decomposed at multiple scales using wavelet transforms. Local extreme points are detected at different scales, and the stable peak position is determined through scale-space analysis. The peak position is used as a candidate for the class boundary. A search window is constructed near the candidate location, and the window size is adaptively adjusted based on the steepness of the probability distribution. An improved dynamic programming algorithm is used to find the optimal threshold combination within the search window. The optimal thresholds for reflection intensity and geometric features are determined through multi-objective optimization by maximizing the between-class variance and minimizing the within-class variance. Information entropy analysis is performed on the histogram to calculate the discriminative power of different features. A weight calculation method based on information gain is used to convert the entropy value into a feature fusion weight. The two optimal thresholds are fused through a weighted combination to achieve feature-adaptive optimal segmentation threshold determination.

[0082] For example, let's take traffic sign point cloud data processing as an example. For each sample point on the sign's surface, the metal material's reflection coefficient is extracted. The atmospheric attenuation coefficient is calculated based on the weather conditions at the time of acquisition. The distance from the lidar to the sign and the angle between the laser beam and the sign's surface are measured to calculate the reflection intensity value for each point in the point cloud data.

[0083] For a sampling point on the sign surface, we select points within a 5-centimeter radius around it to construct a local neighborhood point set. We calculate the eigenvalues ​​of the covariance matrix of this point set to obtain the local curvature value. We also calculate the normal vector between this point and its neighboring points to obtain the normal vector consistency value. Because the sign surface is relatively flat, the local curvature value is small, resulting in a high normal vector consistency value.

[0084] By analyzing the distribution of reflection intensity values ​​in the sign point cloud data and constructing a reflection intensity histogram, we can see that due to the uniformity of the sign surface material, the reflection intensity values ​​are relatively concentrated. By analyzing the distribution characteristics of local curvature values ​​and normal vector consistency values ​​and constructing a geometric feature histogram, we can observe the characteristic differences between flat areas and edge areas. The histogram is normalized.

[0085] By maximizing the inter-class variance, the optimal threshold for distinguishing the sign's main body from its background can be obtained from the reflection intensity histogram. The optimal threshold for distinguishing between planar and edge regions can be obtained from the geometric feature histogram. Considering that the sign's surface reflection features are more significant than its geometric features, the reflection intensity histogram has a larger information entropy. Based on this, a larger reflection intensity feature weight is calculated, resulting in the optimal segmentation threshold that favors the reflection intensity feature.

[0086] In this embodiment, the quality of point cloud data is significantly improved through multi-level data preprocessing and quality control mechanisms, laying a solid foundation for subsequent feature extraction. Based on adaptive neighborhood selection and multi-scale feature extraction strategies, the expression ability of geometric features for local structures is enhanced, and the accuracy and robustness of feature description are improved. The adaptive threshold determination method of multi-objective optimization and information entropy weight is adopted to achieve optimal feature fusion and improve the accuracy and adaptability of the segmentation threshold.

[0087] In an optional embodiment,

[0088] The initial point cloud data is divided into multiple initial sub-regions according to the optimal segmentation threshold, the scattering coefficient and absorption coefficient of each initial sub-region are calculated, and adjacent regions are merged based on spatial distribution characteristics. The obtained point cloud sub-regions with uniform material include:

[0089] Segmenting the initial point cloud data into a plurality of sub-regions based on the optimal segmentation threshold to obtain a plurality of initial sub-regions;

[0090] collecting scattered light intensity, incident light intensity, and transmitted light intensity of the initial sub-area, calculating a scattering coefficient based on a ratio of the scattered light intensity to the incident light intensity, and calculating an absorption coefficient based on a ratio of the transmitted light intensity to the incident light intensity;

[0091] Extracting the number of point cloud data points in the initial subregion, measuring the bounding volume of the initial subregion, calculating the ratio of the number of point cloud data points to the bounding volume to obtain a spatial distribution characteristic, obtaining the spatial distance between any two point cloud data points in the initial subregion, substituting the spatial distance into a Gaussian function to calculate a spatial continuity coefficient, extracting the scattering coefficient and absorption coefficient of adjacent initial subregions to calculate an optical characteristic distance, and calculating the difference between the spatial distribution characteristics and the spatial continuity coefficient of the adjacent initial subregions to obtain a spatial distribution distance;

[0092] Weight coefficients are applied to the optical characteristic distance and the spatial distribution distance respectively and the sum is calculated to obtain a region merging criterion function value. When the region merging criterion function value is less than a preset threshold, the adjacent initial sub-regions are merged to obtain a point cloud sub-region with uniform material. The merging operation is repeated until the region merging criterion function values ​​of all adjacent point cloud sub-regions are greater than the preset threshold.

[0093] The initial point cloud data is loaded into the data processing unit, and the optimal segmentation threshold is applied to segment the point cloud data. An octree index structure is established for the point cloud data to accelerate the region growth process. Unsegmented seed points are selected and the region is expanded around them using the optimal segmentation threshold as the judgment criterion until all point cloud data is allocated, resulting in multiple initial sub-regions. The boundaries of each initial sub-region are optimized to eliminate fragmented areas and smooth the region boundaries.

[0094] For each initial sub-area, multiple light intensity sensor arrays are deployed to collect light intensity data. Positioning is optimized to ensure that the sensor field of view covers the entire sub-area. Intensity data for incident, scattered, and transmitted light are collected separately, with multiple measurements taken in each direction and averaged to eliminate random fluctuations. The collected light intensity data is temperature compensated and background light subtracted to ensure data accuracy. The scattering coefficient is calculated by calculating the ratio of scattered light intensity to incident light intensity, and the absorption coefficient is calculated by calculating the ratio of transmitted light intensity to incident light intensity.

[0095] Spatial feature analysis is performed on each initial subregion, and the total number of point cloud data points within the subregion is counted. A convex hull algorithm is used to calculate the three-dimensional bounding volume of the subregion and obtain its volume. The ratio of the number of point cloud data points to the bounding volume is calculated to obtain spatial distribution characteristics that characterize the point cloud density distribution. A grid-based sampling method is used within the subregion to extract the three-dimensional Euclidean distance between any two point cloud data points. This spatial distance is then substituted into a Gaussian kernel function for mapping, resulting in a spatial continuity coefficient that reflects the continuity of the point cloud data.

[0096] An adjacency graph of adjacent subregions is constructed, and the scattering coefficients and absorption coefficients of each pair of adjacent subregions are extracted. The differences in the scattering coefficients and absorption coefficients between adjacent subregions are calculated and weighted together to obtain the optical characteristic distance. The spatial distribution characteristics and spatial continuity coefficients of adjacent subregions are extracted and their difference is calculated to obtain the spatial distribution distance. Weight coefficients are assigned to the optical characteristic distance and the spatial distribution distance, and the sum of these coefficients is used to obtain the region merging criterion function value.

[0097] Region merging is performed iteratively. The merging criterion function values ​​for all pairs of adjacent sub-regions are sorted. Starting from the minimum value, each pair is evaluated individually. When the function value is less than a preset threshold, the corresponding adjacent sub-regions are merged. After merging, parameters such as the scattering coefficient, absorption coefficient, and spatial distribution characteristics of the region are updated. The merging criterion function value for the new region and its adjacent regions is recalculated. The merging operation continues until the merging criterion function values ​​for all pairs of adjacent regions are greater than the preset threshold, resulting in a point cloud sub-region with uniform material quality.

[0098] For example, in the case of processing point cloud data from a traffic sign, the optimal segmentation threshold is applied to the acquired point cloud data for initial segmentation. Because the sign surface has areas of different colors, the initial segmentation yields multiple sub-regions, including a red border area, a white background area, and a black text area.

[0099] During the light intensity data collection phase, an array of light intensity sensors was placed around the sign. Data was collected in the red border area, measuring an incident light intensity of 100, a scattered light intensity of 60, and a transmitted light intensity of 30. This yielded a calculated scattering coefficient of 0.6 and an absorption coefficient of 0.3. Data was collected in the white background area, measuring an incident light intensity of 100, a scattered light intensity of 80, and a transmitted light intensity of 15. This yielded a calculated scattering coefficient of 0.8 and an absorption coefficient of 0.15.

[0100] Analyzing the spatial characteristics of the red-bordered area revealed a count of 1000 point cloud data points, a calculated bounding volume of 0.1 cubic meters, and a spatial distribution characteristic of 10,000. The distance between any two points within the area was extracted and mapped using a Gaussian function, yielding a spatial continuity coefficient of 0.85. Similarly, analyzing the white-background area revealed a spatial distribution characteristic of 12,000 and a spatial continuity coefficient of 0.9.

[0101] The optical characteristic distance between the red border area and the white background area was calculated to be 0.25, and the spatial distribution distance was 0.15. Setting the optical characteristic weight to 0.6 and the spatial distribution weight to 0.4, the calculated region merging criterion function value was 0.21. Because this value was greater than the preset threshold of 0.2, the two regions were kept separate. Through multiple rounds of iteration, the sign's component regions with stable material properties were finally obtained.

[0102] In this embodiment, the initial region division is performed based on the optimal segmentation threshold, avoiding the limitations of the traditional fixed threshold method. The accuracy of the scattering coefficient and absorption coefficient calculation is ensured through multi-angle light intensity acquisition and data compensation mechanism. The point cloud density distribution is combined with spatial continuity to enhance the ability to express regional structural characteristics. The adaptive weighted criterion function is used to achieve the optimal fusion of optical characteristics and spatial characteristics.

[0103] Existing point cloud data segmentation methods primarily rely on single geometric features or reflection intensity features to segment regions, without considering the differences in material properties. This can lead to over-segmentation or under-segmentation when processing targets with complex materials. Traditional methods often use simple distance thresholds when merging regions, lacking in-depth analysis of the region's physical properties, making it difficult to accurately segment regions with similar geometric structures but different materials.

[0104] This embodiment significantly improves the ability to distinguish regions of different materials by introducing material optical property analysis, effectively solving the segmentation accuracy problem of traditional methods when dealing with targets with complex materials. The regional merging strategy based on multi-dimensional feature fusion improves the accuracy of regional division and avoids the occurrence of over-segmentation and under-segmentation. The iterative optimization merging mechanism ensures the stability and reliability of the final segmentation result, laying a solid foundation for subsequent point cloud data analysis and application.

[0105] Figure 2 This is a spatial distribution feature map of point cloud data for the image-based three-dimensional point cloud traffic sign classification method according to an embodiment of the present invention, with changes in point cloud density represented by different color shades. Through intuitive visualization, the spatial distribution differences between regions of different materials can be clearly identified. The blue area in the figure represents the red border area, with a point cloud density of approximately 10,000 points per cubic meter; the green area represents the white background area, with a density of approximately 12,000 points per cubic meter; and the yellow and red areas represent the black character area, with a density of up to 14,000-16,000 points per cubic meter.

[0106] Density differences reflect the varying reflective properties of different materials to laser scanners. While the black character area is small, the material's characteristics result in a strong laser reflection signal, leading to a denser point cloud. The red border area, on the other hand, has a relatively sparse point cloud, directly related to its optical properties. This technical solution utilizes spatial distribution features combined with optical parameters to determine region merging, resolving issues that are difficult to address with traditional methods based solely on geometric features or statistical clustering.

[0107] The distribution characteristics in the heat map also reveal the advantages of this technical solution when dealing with boundary regions. Traditional region growing methods rely primarily on spatial adjacency to determine boundary regions, which can easily lead to over-segmentation or under-segmentation. K-means clustering, on the other hand, cannot effectively utilize spatial continuity information. By introducing spatial distribution characteristics and spatial continuity coefficients, this technical solution can more accurately determine material boundaries, particularly in regions with similar optical properties but significantly different spatial distribution characteristics.

[0108] In an optional embodiment,

[0109] Calculating the local curvature entropy and normal vector entropy of the point cloud sub-regions, ranking the importance using the entropy values ​​as weight coefficients, determining the core region, and establishing a local coordinate system based on the core region includes:

[0110] Select a target point in the point cloud sub-region, construct a spherical search region with a preset radius with the target point as the center, extract neighborhood points in the spherical search region to form a local neighborhood point set, and calculate the covariance matrix of the local neighborhood point set;

[0111] Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​arranged in descending order, taking the ratio of the minimum eigenvalue to the sum of the eigenvalues ​​as the local curvature of the target point, traversing all point cloud data points in the point cloud sub-region to repeatedly calculate the local curvature to obtain a local curvature set, and statistically calculating the distribution probability of the local curvature set within a preset interval to obtain a curvature entropy value;

[0112] Mapping the normal vectors of the point cloud data points in the point cloud sub-region to a unit sphere and dividing the sphere into a plurality of direction intervals, and calculating the normal vector distribution probability in each direction interval to obtain a normal vector entropy value;

[0113] Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value, dividing the comprehensive entropy value by the maximum comprehensive entropy value of all point cloud sub-regions to obtain an importance coefficient, calculating the volume of the point cloud sub-region, and taking the product of the importance coefficient and the volume as the region sorting criterion value, and selecting the point cloud sub-region with the maximum region sorting criterion value as the core region;

[0114] The covariance matrix of the core area is calculated and the eigenvector is extracted to determine the direction of the coordinate axis. The center of mass position of the core area is calculated and the coordinate origin is obtained by offsetting the eigenvector direction corresponding to the maximum eigenvalue by a preset distance to establish a local coordinate system.

[0115] Target points are selected within a point cloud subregion, using a uniform sampling strategy to select representative target points. A spherical search region is constructed for each target point, with the search radius adaptively adjusted based on the point cloud density. A spatial index structure is used to quickly retrieve neighboring points within the spherical region, removing noise points to construct a local neighborhood point set. The neighborhood point set is then centered, and the covariance matrix of the point set is calculated, which reflects the spatial distribution characteristics of the local point cloud.

[0116] The calculated covariance matrix is ​​subjected to eigenvalue decomposition, and the three eigenvalues ​​are obtained and sorted from largest to smallest. The ratio of the smallest eigenvalue to the sum of the eigenvalues ​​is calculated to obtain the geometric features that characterize the local curvature. Through multi-threaded parallel computing, all data points in the point cloud subregion are traversed, and the local curvature calculation process is repeated to establish a local curvature set. The curvature values ​​are divided into multiple intervals, and the frequency of curvature values ​​within each interval is counted. The probability distribution is calculated to obtain the curvature entropy value.

[0117] Extract the normal vector information of the point cloud data points and map all normal vector endpoints onto the unit sphere. Adaptive meshing is used to divide the unit sphere into multiple directional intervals, with interval sizes dynamically adjusted based on the normal vector distribution density. The number of normal vectors in each directional interval is counted, and the probability distribution of each interval is calculated to obtain the normal vector entropy.

[0118] A feature fusion framework is constructed based on the dynamic manifold learning method, and the curvature entropy and normal vector entropy are mapped to the manifold space. Geometric persistence constraints are considered in the manifold space to establish the correlation between features. The optimal fusion weight is determined through iterative optimization, and the curvature entropy and normal vector entropy are adaptively fused to obtain the comprehensive entropy value. The comprehensive entropy values ​​of all point cloud sub-regions are searched, and the maximum value is found as the normalization benchmark. The comprehensive entropy value of each sub-region is divided by the maximum value to obtain the importance coefficient. The volume of the three-dimensional bounding box of the point cloud sub-region is measured, and the importance coefficient is multiplied by the volume to obtain the region ranking criterion value. The ranking criterion values ​​of all sub-regions are compared, and the point cloud sub-region corresponding to the maximum value is selected as the core region.

[0119] Perform a principal direction analysis on the core area, calculate the covariance matrix of the regional point cloud, and extract eigenvectors. The eigenvectors reflect the primary direction of the point cloud distribution, and the eigenvector corresponding to the largest eigenvalue is selected as the principal axis direction. Calculate the center of mass of all points in the core area, move the center of mass a preset distance along the principal axis, and determine the origin of the coordinate system. Establish a local coordinate system based on the principal axis direction and the origin, completing the orientation and positioning of the coordinate system.

[0120] For example, using the point cloud data processing of traffic signs in a road scene, evenly distributed target points are selected within the sign area. A spherical search region is constructed for a target point on the sign surface. Because the sign surface is flat, the search radius is set to one-tenth of the sign size. A set of neighboring points is extracted within the spherical region. Because the sign surface is a regular plane, the spatial distribution of the neighboring points is relatively regular.

[0121] Due to the flat surface features of standard traffic signs, the calculated local curvature values ​​are generally small. At the edges of the sign, due to a sudden change in the point cloud distribution, the corresponding local curvature values ​​are relatively large. By traversing all the point cloud data within the sign area, the calculated curvature set exhibits a clear bimodal distribution, reflecting the structural differences between the flat and edge regions. The resulting curvature entropy value effectively characterizes the sign's geometric characteristics.

[0122] The normal vectors of the sign point cloud are generally perpendicular to the road. Mapping all normal vectors onto the unit sphere reveals a highly concentrated distribution. In the sphere's interval divisions, the intervals perpendicular to the road contain the vast majority of normal vectors, while the number of normal vectors in intervals other than the road is minimal. This distribution leads to a low calculated normal vector entropy, reflecting the sign's regular orientation.

[0123] Feature fusion is used to determine the comprehensive entropy of the sign area. Due to the regular geometry and uniform installation orientation of standard traffic signs, their comprehensive entropy is lower than that of other roadside objects. Given the fixed dimensions of signs, their volume remains within the standard range. The comprehensive entropy is multiplied by the volume to determine the region ranking criterion. The sign area is selected as the core region due to its typical geometric and directional characteristics.

[0124] The core area of ​​the sign is analyzed for principal directions, and the extracted feature vectors align with the vertical edge of the sign. The center of mass of the sign is calculated, and the coordinate origin is determined by vertically offsetting the sign by a preset distance. This local coordinate system is then established with the sign as the reference. This coordinate system can be used for subsequent sign posture assessment and recognition and classification tasks. This local coordinate system provides a reference for further analysis of state parameters such as the sign's tilt angle and steering angle.

[0125] In this embodiment, accurate extraction of local features of point cloud data is achieved through target point selection and local neighborhood construction. The geometric features of traffic signs are comprehensively characterized through the dual feature extraction mechanisms of local curvature calculation and normal vector analysis. The geometric complexity of the sign surface is accurately expressed through the calculation of curvature entropy values. The feature fusion strategy based on dynamic manifold learning and geometric persistence compensation realizes the optimized combination of curvature features and directional features, thereby improving the comprehensiveness of feature expression.

[0126] In an optional embodiment,

[0127] Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value includes:

[0128] Constructing a local connection graph for the local neighborhood point set and calculating the geodesic distance between any two neighborhood points, projecting the local neighborhood point set into a manifold space, and calculating the mean curvature and Gaussian curvature based on the geodesic distance;

[0129] Calculating a first principal curvature and a second principal curvature based on the mean curvature and the Gaussian curvature, taking a negative exponential of a product of an absolute value of the Gaussian curvature and a preset attenuation factor, and multiplying the product by the difference between the first principal curvature and the second principal curvature plus one to obtain a dynamic manifold weight;

[0130] Extracting the eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating the curvature change rate along the principal curvature direction based on the eigenvalues ​​and the first principal curvature and the second principal curvature to obtain a geometric structure eigenvalue, calculating the geometric structure eigenvalue angle and spatial distance of adjacent local neighborhood points, multiplying the cosine value of the angle by the exponential decay value of the spatial distance to construct a persistence matrix, and calculating the ratio of the maximum eigenvalue of the persistence matrix to the matrix trace to obtain a geometric persistence coefficient;

[0131] Calculate the variance of the geometric structure characteristic value, multiply the preset compensation coefficient by the inverse of the geometric persistence coefficient plus one, and multiply the variance plus the logarithm of one to obtain a dynamic entropy compensation value, multiply the dynamic manifold weight by the weighted sum of the curvature entropy value and the normal vector entropy value, and superimpose the dynamic entropy compensation value to obtain a comprehensive entropy value.

[0132] For a local neighborhood point set, a spherical search range is used to determine the nearest neighbor point set for each point. A local connection graph is constructed based on the nearest neighbor relationship, with undirected edges connecting each point to its nearest neighbors. The Dijkstra algorithm is used to calculate the shortest path length between any two points in the connection graph, and this length is used as the geodesic distance between the point pairs. To improve computational efficiency, a multi-threaded parallel processing strategy is used to distribute the point-pair distance calculation task across multiple processing units.

[0133] A local linear embedding algorithm is used to project a set of neighborhood points into manifold space. A weight matrix for the neighborhood points is constructed, and the projected coordinates that preserve local structure are obtained by solving a system of linear equations. In the manifold space, a local surface fit is constructed based on geodesic distances, and the mean curvature of each point is calculated using the moving least squares method. Gaussian curvature is obtained by constructing a local coordinate system and calculating the second fundamental form of the surface. To ensure the stability of the curvature calculation, neighborhood points are weighted, with points closer to the target point receiving greater weights.

[0134] After obtaining the mean and Gaussian curvatures, the Newton iteration method is used to solve the characteristic equation and calculate the first and second principal curvatures of the local surface. An adaptive attenuation factor is introduced, which dynamically adjusts as the Gaussian curvature changes, providing greater attenuation in areas with drastic curvature changes. The curvature attenuation term is obtained by multiplying the absolute value of the Gaussian curvature by the attenuation factor and performing a negative exponential conversion. The difference between the first and second principal curvatures is calculated, added to one, and multiplied by the curvature attenuation term to construct a dynamic manifold weight that reflects the local surface characteristics.

[0135] Principal component analysis is performed on the local neighborhood point set to construct a covariance matrix and extract its eigenvalues. The eigenvalue decomposition method is used to determine the primary and secondary directions of the point set. The curvature change rate is calculated along the principal curvature direction, and the curvature is discretely differentiated using the finite difference method to obtain the eigenvalues ​​that represent the local geometric structure. The eigenvalue directions of adjacent local neighborhoods are analyzed, and the directional differences of the eigenvalues ​​are obtained using the vector angle calculation method. The Euclidean distance between the center points of adjacent local neighborhoods is also calculated, and the distance decay is performed using a Gaussian function. The cosine of the direction angle is multiplied by the distance decay value to construct a persistence matrix that describes the continuity of the geometric features of the local area.

[0136] The persistence matrix is ​​subjected to eigenvalue decomposition, and the maximum eigenvalue is calculated using the power iteration method. The matrix trace is then calculated by summing all matrix elements. The degree of geometric persistence in a local region is quantified by the ratio of the maximum eigenvalue to the matrix trace. The statistical variance of the geometric structure eigenvalues ​​of all local neighborhoods is calculated, and the local variance is estimated using a sliding window method to improve computational efficiency. The dynamic entropy compensation value is obtained by multiplying the preset compensation coefficient with the complementary value of the geometric persistence coefficient and then multiplying it by the logarithm of the eigenvalue variance.

[0137] An adaptive weighting scheme is used to perform a weighted fusion of curvature entropy and normal vector entropy. The weight coefficients are dynamically adjusted based on the geometric complexity of the local area, increasing the weight of normal vector entropy in structurally simple areas and increasing the weight of curvature entropy in structurally complex areas. The weighted sum is multiplied by the dynamic manifold weight and superimposed with a dynamic entropy compensation value to obtain a comprehensive entropy value. Parameters in each calculation step are adjusted through iterative optimization to ensure the stability and reliability of the results.

[0138] For example, using point cloud data processing for a speed limit sign in a road scene, a local neighborhood point set on the sign surface was selected. When constructing a local connectivity graph, each point was connected to its eight nearest surrounding points. Within the flat area of ​​the sign, the difference between the calculated geodesic distance and the Euclidean distance was within millimeters. However, the geodesic distance increased significantly at the edges of the digits and the sign border, reflecting changes in the local structure.

[0139] Projecting the neighborhood point set onto a two-dimensional manifold space reveals that the average curvature of the sign's main planar surface is close to zero, within a range of 0.001. At the edges of the digits, the average curvature increases to a range of 0.01 to 0.05. Gaussian curvature also approaches zero in the planar surface, with local peaks reaching as high as 0.1 at the edges.

[0140] The calculated first and second principal curvatures are both close to zero in planar regions, with a difference of no more than 0.001. At the digital edges and border transitions, the two principal curvatures show significant differences, ranging from 0.05 to 0.1. The dynamic manifold weight remains high, above 0.9, in planar regions and decreases to a range of 0.3 to 0.5 at the edges, effectively distinguishing regions with different structural characteristics.

[0141] Covariance analysis of the sign surface shows that in flat areas, the ratio of the three eigenvalues ​​is close to 100:100:1, reflecting a standard flat surface. At the edges of the digits, the eigenvalue ratio changes to approximately 100:10:1, reflecting the linear structure of the edge. The calculated geometric structure eigenvalues ​​vary smoothly in flat areas, with the angle between eigenvalues ​​in adjacent areas less than 5 degrees. At the edges of the digits and borders, the eigenvalue angle increases to over 30 degrees, and the exponential decay with spatial distance decreases accordingly.

[0142] The resulting comprehensive entropy value remains low, below 0.1, on the flat surface of the sign, rising to between 0.3 and 0.5 around the edges of the digits and the border transition area, reaching a maximum of around 0.8 at the corners. This entropy distribution accurately reflects the structural differences between the sign's various regions, providing a reliable feature index for subsequent feature extraction and classification.

[0143] In this embodiment, the local topological structure of the point cloud data is accurately described by constructing a local connection graph and calculating geodesic distances. The manifold space projection technique is used to effectively maintain the local structural relationship of the data, laying the foundation for subsequent curvature feature extraction. The precise calculation of mean curvature and Gaussian curvature, combined with principal curvature analysis, achieves a comprehensive characterization of local surface features. The introduction of a dynamic manifold weight mechanism enables the feature extraction process to adaptively adjust according to changes in curvature in the local area, improving the robustness of feature extraction.

[0144] Existing technologies primarily rely on simple geometric feature calculations during point cloud feature extraction. For example, single features such as curvature statistics or normal vector distribution within a fixed neighborhood are used. These methods are unable to accurately describe the local details of traffic signs in complex scenes. Feature fusion often uses a linear combination with fixed weights, which fails to fully consider the structural differences in local areas and results in insufficient adaptability of feature expression.

[0145] This embodiment improves the accuracy of local feature extraction through the combination of geodesic distance and manifold projection, and can effectively express the local structural features of complex surfaces. The introduction of dynamic manifold weights realizes adaptive adjustment of feature extraction and enhances the algorithm's adaptability to different types of landmarks. The feature fusion strategy based on geometric persistence analysis improves the distinguishability of feature expression and effectively solves the continuity problem of local area feature transition.

[0146] Figure 3 This is a comparison chart of the principal curvature calculation accuracy of the image-based three-dimensional point cloud traffic sign classification method according to an embodiment of the present invention, showing comparative data on the principal curvature calculation accuracy of different methods.

[0147] This technical solution performs well in calculating the principal curvature of various surfaces through dynamic manifold learning and geometric persistence compensation mechanism.

[0148] In standard sphere tests, the error rate of this technical solution was only 0.32%, a reduction of 82.8% compared to the 1.86% of the traditional PCA method and a 66.3% reduction compared to the 0.95% of the Moving Least Square method. For planar structures, the error rate of this technical solution was as low as 0.08%, a reduction of 66.7% and 52.9% compared to the 0.24% of the traditional method and 0.17% of the MLS method, respectively. Most notably, when processing sharp edges, the error rate of this technical solution was 2.83%, far lower than the 9.73% of the PCA method and 6.54% of the MLS method, resulting in accuracy improvements of more than 70.9% and 56.7%.

[0149] For the edge detection of digital signs in real-world applications, this solution achieved an error rate of 1.47%, a 61.5% improvement in accuracy compared to other methods. The figure shows that the performance gap between this solution and existing methods becomes more pronounced with increasing geometric complexity, particularly when dealing with non-uniform sampling points and areas with high curvature variations.

[0150] In an optional embodiment,

[0151] The region growth range is determined by combining the spatial position relationship and the geometric structure relationship, and the regions that meet the pre-set constraints are selected for dynamic merging. The candidate point cloud sub-regions include:

[0152] For each point in the point cloud sub-region, the minimum Euclidean distance between the current point and the core region is calculated to obtain a spatial position relationship, and the dot product of the normal vector of the current point and the average normal vector of the core region is calculated to obtain a geometric structure relationship;

[0153] Performing a product operation on the spatial position relationship and the complement of the geometric structure relationship to obtain a growth constraint value, and determining an area in the point cloud sub-area where the growth constraint value is less than a preset growth threshold as a regional growth range;

[0154] Calculating the shortest distance between adjacent regions within the region growth range to obtain an inter-region distance threshold, calculating the dot product of the normal vectors of the adjacent regions to obtain normal vector consistency, calculating the curvature difference of the adjacent regions to obtain curvature continuity, and performing a weighted combination of the inter-region distance threshold, the normal vector consistency, and the curvature continuity to obtain region similarity;

[0155] Adjacent regions whose region similarity is greater than a preset merging threshold are dynamically merged, the boundary features and region feature parameters of the merged region are updated, and the merging operation is repeated until the region similarity is less than the preset merging threshold to obtain a candidate point cloud sub-region.

[0156] Perform positional and structural analysis on each data point within the point cloud subregion. Establish a point cloud index tree structure for the core region to accelerate the nearest neighbor search process. Traverse each data point and use the index tree to quickly find the distance from that point to all points in the core region. Select the minimum distance as the spatial position relationship indicator. Calculate the average normal vector for all points in the core region and normalize it to ensure it is a unit vector. Calculate the dot product of the current point's normal vector with the average normal vector to obtain the geometric structure relationship indicator.

[0157] After obtaining the spatial position and geometric structure relationships, the geometric structure relationship is complemented and multiplied with the spatial position relationship to obtain the growth constraint value. To improve computational efficiency, a multi-threaded parallel processing strategy is used to simultaneously calculate the growth constraint values ​​for multiple points. A preset growth threshold is set, and point cloud regions with growth constraint values ​​below the threshold are marked as regional growth ranges. Connected domain analysis is used to divide the growth range to ensure regional continuity.

[0158] Within the resulting region growth range, all adjacent region pairs are identified. Bounding boxes are constructed to quickly identify potentially adjacent regions, and true adjacency is determined through point-to-point distance calculations. For each pair of adjacent regions, multiple sampling point pairs are extracted using a grid-based sampling method within the boundary region. The shortest distance between these sampling point pairs is calculated, and the minimum value is selected as the inter-region distance threshold.

[0159] The normal vectors of all points within adjacent regions are counted, and the average dot product of the normal vectors of the two regions is calculated as a normal consistency indicator. The mean and variance of the point cloud curvature are calculated for each region, and curvature continuity is assessed based on the difference in means and the degree of overlap in variances. An adaptive weighting scheme is used to dynamically adjust the weight coefficients of the three indicators (distance threshold, normal consistency, and curvature continuity) based on the geometric characteristics of the region. These three weighted indicators are combined to determine the region similarity.

[0160] Establish a priority queue for region merging and sort the regions in descending order of similarity. Each time, the region pair with the highest similarity is taken from the head of the queue and merged. A boundary smoothing strategy is used during merging to ensure the continuity of the merged region boundaries. Update the feature parameters of the merged region, including recalculating the region's normal vector distribution and curvature characteristics. Also, update the adjacency and similarity between the merged region and other regions. The updated similarity is reinserted into the priority queue to maintain order in the queue.

[0161] The merging operation is repeated until the maximum region similarity in the queue falls below a preset merging threshold. A union-find data structure is used to record the region merging process, facilitating tracking of the region's evolutionary history. The resulting candidate point cloud subregions possess similar geometric features and a continuous spatial distribution.

[0162] For example, consider a traffic sign combination at an intersection, consisting of a speed limit sign and a directional sign. After preliminary processing, the circular area of ​​the speed limit sign is selected as the core area. The surrounding point cloud data is analyzed to calculate the minimum distance from each point to the speed limit sign's circular disc. The distances between points on the sign pole are relatively large, exceeding 20 centimeters, while the distances between points on adjacent signs are relatively small, approximately 10 centimeters.

[0163] The average normal vector of the core area was calculated. Due to the standard installation of speed limit signs, its normal vector was essentially perpendicular to the road. The dot product of the normal vectors of the surrounding points was calculated. Since the directional signs were installed on the same signpost, their dot product was close to 1, while the dot product of the normal vectors of the points on the signpost was close to 0. The calculation results of the growth constraint values ​​showed that the constraint values ​​of the directional sign area were generally less than 0.3, while the constraint values ​​of the signpost area were greater than 0.7.

[0164] Within the defined growth range, the directional markers were divided into several initial subregions. The shortest distance between adjacent subregions was generally less than 5 mm, the dot product of the normal vectors was greater than 0.95, and the curvature difference was less than 0.01. These characteristics indicate a high degree of similarity between adjacent subregions. After weighted combination, the calculated regional similarity was greater than 0.8.

[0165] Through multiple rounds of iterative merging, the multiple subregions of the directional sign are gradually merged into a complete surface area. During this merging process, the normal vectors and curvature characteristics of the boundary points are smoothly transitioned. The resulting candidate point cloud subregions encompass the entire directional sign surface, with clear boundaries and continuous geometric features, forming a reasonable spatial layout relationship with the core area of ​​the speed limit sign.

[0166] In this embodiment, a reasonable region growing range definition mechanism is established through the joint constraints of spatial position relationships and geometric structure relationships. When calculating the spatial position relationship, the minimum Euclidean distance is used as the evaluation indicator to accurately describe the spatial proximity between the processing point and the core area. The geometric structure relationship is calculated by the dot product of the normal vector, which effectively characterizes the structural consistency of the local area.

[0167] Existing point cloud region growing methods primarily rely on simple distance thresholds and normal vector differences for region expansion, lacking comprehensive consideration of inter-region structural relationships. Fixed merging criteria are often used during region merging, failing to adapt to the structural characteristics of different types of traffic signs. This can easily lead to over-merging or under-segmentation, and makes it difficult to accurately identify and maintain spatial correlations between multiple adjacent signs.

[0168] The core region-guided growth strategy of this embodiment significantly improves the accuracy and controllability of region growth, and can effectively identify continuous regions with similar structural characteristics. The multi-dimensional region similarity evaluation mechanism enhances the adaptability of region merging and effectively avoids unreasonable region merging. The dynamically updated feature parameters ensure the continuity and stability of the region merging process.

[0169] Figure 4 This figure illustrates the point cloud region growing process for the image-based 3D point cloud traffic sign classification method according to an embodiment of the present invention. It shows the entire process of region growing for the circular portion of a traffic sign. The white dots represent the original point cloud data, forming a circular outline with an initial radius of approximately 80 mm, corresponding to the basic shape of the speed limit sign. The red dots represent the point cloud data after expansion using the region growing algorithm, ultimately forming a complete circular region with a radius of approximately 120 mm.

[0170] The blue boundary line marks the initial region, while the green boundary line shows the final boundary after region growing. The four arrows indicate the main directions of region growing. The point cloud expands evenly in all directions, maintaining the original circular geometry.

[0171] The region growing process is based on strict geometric constraints, including a normal vector consistency of 0.96 (indicating that the surface orientation of the expanded region is highly consistent with that of the initial region) and a curvature continuity of 0.03 (indicating very small changes in curvature). The combined growth constraint value is 0.21, which is lower than the preset threshold, ensuring that the grown region has consistent geometric properties with the original region.

[0172] Through the region growing method based on geometric structure constraints, the circular part of the traffic sign can be accurately identified and expanded. While maintaining the original geometric features, it makes up for the discontinuities or missing parts that may exist in the original point cloud data, providing more complete and accurate geometric information for subsequent sign recognition and three-dimensional reconstruction.

[0173] In an optional embodiment,

[0174] Extract the position offset and attitude change of the candidate point cloud sub-region in adjacent multi-frame data, construct the motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier, and obtain the traffic sign classification results including:

[0175] Obtain the spatial coordinates of the candidate point cloud subregion in adjacent multi-frame point cloud data, calculate the coordinate difference between adjacent frames to obtain a position offset matrix, use the coordinate axis direction of the local coordinate system as the main axis direction, calculate the Euler angle change of the candidate point cloud subregion between adjacent frames to obtain a posture change matrix;

[0176] The projection component of the position offset matrix in three-dimensional space and the pitch angle, roll angle, and yaw angle changes of the attitude change matrix are combined into a six-dimensional vector, a motion state descriptor based on a sliding time window is constructed, principal component analysis is performed on the temporal change characteristics of the motion state descriptor to extract the change pattern of the key dimension, and the change patterns in multiple time windows are cascaded to obtain a target feature vector;

[0177] The target feature vector is input into a pre-trained support vector classifier to obtain a traffic sign classification result.

[0178] Candidate point cloud subregions are extracted from continuously acquired multi-frame point cloud data, and the spatial coordinates of the subregion's center of mass are recorded in each frame. Inter-frame differencing is used to calculate the coordinate changes between adjacent frames, constructing a position offset matrix that reflects the target's motion trend. Based on the established local coordinate system, the coordinate axis directions are used as a reference to calculate the rotational changes of the candidate regions between adjacent frames. By decomposing the rotation matrix, the angular changes in the target's posture in three degrees of freedom are obtained, and a posture change matrix is ​​constructed.

[0179] The position offset matrix is ​​decomposed into projected components along the three coordinate axes. Combined with the three angular variations in the attitude change matrix, this creates a six-dimensional vector describing the target's motion state. The length of the sliding time window is set, and the motion state vectors of multiple consecutive frames are combined into a time series data matrix. The data within each time window is mean-centered and normalized to eliminate scale differences between different dimensions.

[0180] Perform principal component analysis on the processed time series data matrix to extract the main patterns of data variation. By calculating eigenvalues ​​and eigenvectors, identify the key dimensions that contribute most to the motion characteristics. Sort the eigenvectors by eigenvalue, and select the dimensions with the highest contribution as the primary features. Repeat principal component analysis for multiple consecutive time windows to identify patterns of variation across different time periods.

[0181] A feature concatenation strategy is used to combine the change patterns extracted from multiple time windows in chronological order to form a complete target feature vector. During the feature concatenation process, the overlapping regions between time windows are considered to ensure the continuity of feature changes. The concatenated feature vector is normalized to eliminate dimensionality effects.

[0182] The normalized target feature vector is fed into a pre-trained support vector classifier. During training, the classifier has learned the motion patterns of different types of traffic signs. A kernel function is used to map the features into a high-dimensional space. The distance between the feature vector and the classification hyperplane is calculated to determine the class to which the sample belongs. The classifier then outputs the traffic sign classification result.

[0183] For example, a vehicle-mounted LiDAR (lidar) continuously collects multiple frames of point cloud data, using a speed limit sign and a yield sign in a road scene. Candidate point cloud subregions of the speed limit sign are extracted, and their motion changes between adjacent frames are recorded. Due to the vibration and bumps of the vehicle during driving, the speed limit sign exhibits slight positional shifts and attitude changes in the local coordinate system.

[0184] Calculating the position change between adjacent frames revealed that the speed limit sign exhibited minimal vertical positional deviation, while exhibiting periodic horizontal variations. Analysis of the attitude change matrix revealed relatively large variations in the speed limit sign's pitch and yaw angles, while small variations in its roll angle. This motion characteristic is consistent with the sign's rigid mounting.

[0185] The motion data was analyzed by selecting fixed-length time windows, each containing multiple frames of data. Principal component analysis of the six-dimensional motion state vector within the window revealed that horizontal displacement and pitch angle changes constituted the primary variation patterns. This characteristic reflects the typical motion characteristics of speed limit signs during vehicle movement.

[0186] The features of multiple time windows are concatenated to obtain a complete feature vector. After inputting it into the support vector classifier, the system accurately identifies the target as a speed limit sign.

[0187] In this embodiment, accurate classification of traffic signs is achieved through dynamic feature analysis of multi-frame point cloud data. By extracting the spatial coordinates of candidate point cloud sub-regions and calculating inter-frame differences, the motion trend of the target is accurately captured. The construction of the position offset matrix reflects the displacement characteristics of the target in space, and the posture change analysis based on the local coordinate system reflects the rotation characteristics of the target. It can effectively handle the interference caused by vehicle movement, provide stable and reliable classification results, and provide strong support for the development of intelligent transportation systems.

[0188] A second aspect of an embodiment of the present invention provides an image-based three-dimensional point cloud traffic sign classification system, comprising:

[0189] The first unit is used to collect continuous multi-frame multi-angle image data of the traffic sign to be classified and obtain initial point cloud data based on camera parameters;

[0190] The second unit is configured to calculate the reflection intensity value and local geometric characteristic value of each point in the initial point cloud data, construct a reflection intensity histogram and a geometric characteristic histogram, and determine an optimal segmentation threshold. The initial point cloud data is divided into a plurality of initial sub-regions according to the optimal segmentation threshold, and the scattering coefficient and absorption coefficient of each initial sub-region are calculated. Adjacent regions are merged based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material.

[0191] The third unit is used to calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select and dynamically merge the regions that meet the pre-set constraints to obtain candidate point cloud sub-regions;

[0192] The fourth unit is used to extract the position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data, construct a motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier to obtain the traffic sign classification result.

[0193] According to a third aspect of the embodiments of the present invention,

[0194] An electronic device is provided, comprising:

[0195] processor;

[0196] a memory for storing processor-executable instructions;

[0197] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0198] According to a fourth aspect of the embodiments of the present invention,

[0199] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0200] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional point cloud traffic sign classification method based on image, characterized by: include: Collect continuous multi-frame and multi-angle image data of traffic signs to be classified, and obtain initial point cloud data based on camera parameters; Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing a reflection intensity histogram and a geometric feature histogram and determining an optimal segmentation threshold, dividing the initial point cloud data into multiple initial sub-regions according to the optimal segmentation threshold, calculating the scattering coefficient and absorption coefficient of each initial sub-region, and merging adjacent regions based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material; Calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select the regions that meet the pre-set constraints and dynamically merge them to obtain candidate point cloud sub-regions; The position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data are extracted, the motion state descriptor is constructed and time series association is performed, the target feature vector is obtained and input into the classifier to obtain the traffic sign classification result.

2. The method according to claim 1, characterized in that Calculating the reflection intensity value and local geometric feature value of each point in the initial point cloud data, constructing the reflection intensity histogram and geometric feature histogram, and determining the optimal segmentation threshold includes: Acquire initial point cloud data comprising a plurality of point cloud data points, extract the material reflection coefficient, atmospheric attenuation coefficient, laser propagation distance, and incident angle of each point cloud data point in the initial point cloud data, and calculate a reflection intensity value; Constructing a local neighborhood point set for each point cloud data point, extracting eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating local curvature values ​​based on the eigenvalues, obtaining normal vectors between the point cloud data point and each point in the local neighborhood point set, and calculating consistency values ​​of the normal vectors; Counting the distribution characteristics of the reflection intensity values ​​to construct a reflection intensity histogram, counting the distribution characteristics of the local curvature values ​​and the consistency values ​​of the normal vectors to construct a geometric feature histogram, and calculating the normalized probability distribution of the reflection intensity histogram and the geometric feature histogram; Based on the normalized probability distribution, the inter-class variance is calculated, and the optimal threshold of reflection intensity and the optimal threshold of geometric features are obtained by maximizing the inter-class variance; the information entropy features of the reflection intensity histogram and the geometric feature histogram are extracted to calculate the feature fusion weight, and the feature fusion weight is applied to the optimal threshold of reflection intensity and the optimal threshold of geometric features respectively to obtain the optimal segmentation threshold.

3. The method according to claim 1, characterized in that The initial point cloud data is divided into multiple initial sub-regions according to the optimal segmentation threshold, the scattering coefficient and absorption coefficient of each initial sub-region are calculated, and adjacent regions are merged based on spatial distribution characteristics. The obtained point cloud sub-regions with uniform material include: Segmenting the initial point cloud data into a plurality of sub-regions based on the optimal segmentation threshold to obtain a plurality of initial sub-regions; collecting scattered light intensity, incident light intensity, and transmitted light intensity of the initial sub-area, calculating a scattering coefficient based on a ratio of the scattered light intensity to the incident light intensity, and calculating an absorption coefficient based on a ratio of the transmitted light intensity to the incident light intensity; Extracting the number of point cloud data points in the initial subregion, measuring the bounding volume of the initial subregion, calculating the ratio of the number of point cloud data points to the bounding volume to obtain a spatial distribution characteristic, obtaining the spatial distance between any two point cloud data points in the initial subregion, substituting the spatial distance into a Gaussian function to calculate a spatial continuity coefficient, extracting the scattering coefficient and absorption coefficient of adjacent initial subregions to calculate an optical characteristic distance, and calculating the difference between the spatial distribution characteristics and the spatial continuity coefficient of the adjacent initial subregions to obtain a spatial distribution distance; Weight coefficients are applied to the optical characteristic distance and the spatial distribution distance respectively and the sum is calculated to obtain a region merging criterion function value. When the region merging criterion function value is less than a preset threshold, the adjacent initial sub-regions are merged to obtain a point cloud sub-region with uniform material. The merging operation is repeated until the region merging criterion function values ​​of all adjacent point cloud sub-regions are greater than the preset threshold.

4. The method according to claim 1, wherein Calculating the local curvature entropy and normal vector entropy of the point cloud sub-regions, ranking the importance using the entropy values ​​as weight coefficients, determining the core region, and establishing a local coordinate system based on the core region includes: Select a target point in the point cloud sub-region, construct a spherical search region with a preset radius with the target point as the center, extract neighborhood points in the spherical search region to form a local neighborhood point set, and calculate the covariance matrix of the local neighborhood point set; Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​arranged in descending order, taking the ratio of the minimum eigenvalue to the sum of the eigenvalues ​​as the local curvature of the target point, traversing all point cloud data points in the point cloud sub-region to repeatedly calculate the local curvature to obtain a local curvature set, and statistically calculating the distribution probability of the local curvature set within a preset interval to obtain a curvature entropy value; Mapping the normal vectors of the point cloud data points in the point cloud sub-region to a unit sphere and dividing the sphere into a plurality of direction intervals, and calculating the normal vector distribution probability in each direction interval to obtain a normal vector entropy value; Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value, dividing the comprehensive entropy value by the maximum comprehensive entropy value of all point cloud sub-regions to obtain an importance coefficient, calculating the volume of the point cloud sub-region, and taking the product of the importance coefficient and the volume as the region sorting criterion value, and selecting the point cloud sub-region with the maximum region sorting criterion value as the core region; The covariance matrix of the core area is calculated and the eigenvector is extracted to determine the direction of the coordinate axis. The center of mass position of the core area is calculated and the coordinate origin is obtained by offsetting the eigenvector direction corresponding to the maximum eigenvalue by a preset distance to establish a local coordinate system.

5. The method according to claim 4, characterized in that Adaptively fusing the curvature entropy value and the normal vector entropy value based on dynamic manifold learning and geometric persistence compensation to obtain a comprehensive entropy value includes: Constructing a local connection graph for the local neighborhood point set and calculating the geodesic distance between any two neighborhood points, projecting the local neighborhood point set into a manifold space, and calculating the mean curvature and Gaussian curvature based on the geodesic distance; Calculating a first principal curvature and a second principal curvature based on the mean curvature and the Gaussian curvature, taking a negative exponential of a product of an absolute value of the Gaussian curvature and a preset attenuation factor, and multiplying the product by the difference between the first principal curvature and the second principal curvature plus one to obtain a dynamic manifold weight; Extracting the eigenvalues ​​of the covariance matrix of the local neighborhood point set, calculating the curvature change rate along the principal curvature direction based on the eigenvalues ​​and the first principal curvature and the second principal curvature to obtain a geometric structure eigenvalue, calculating the geometric structure eigenvalue angle and spatial distance of adjacent local neighborhood points, multiplying the cosine value of the angle by the exponential decay value of the spatial distance to construct a persistence matrix, and calculating the ratio of the maximum eigenvalue of the persistence matrix to the matrix trace to obtain a geometric persistence coefficient; Calculate the variance of the geometric structure characteristic value, multiply the preset compensation coefficient by the inverse of the geometric persistence coefficient plus one, and multiply the variance plus the logarithm of one to obtain a dynamic entropy compensation value, multiply the dynamic manifold weight by the weighted sum of the curvature entropy value and the normal vector entropy value, and superimpose the dynamic entropy compensation value to obtain a comprehensive entropy value.

6. The method according to claim 1, characterized in that The region growth range is determined by combining the spatial position relationship and the geometric structure relationship, and the regions that meet the pre-set constraints are selected for dynamic merging. The candidate point cloud sub-regions include: For each point in the point cloud sub-region, the minimum Euclidean distance between the current point and the core region is calculated to obtain a spatial position relationship, and the dot product of the normal vector of the current point and the average normal vector of the core region is calculated to obtain a geometric structure relationship; Performing a product operation on the spatial position relationship and the complement of the geometric structure relationship to obtain a growth constraint value, and determining an area in the point cloud sub-area where the growth constraint value is less than a preset growth threshold as a regional growth range; Calculating the shortest distance between adjacent regions within the region growth range to obtain an inter-region distance threshold, calculating the dot product of the normal vectors of the adjacent regions to obtain normal vector consistency, calculating the curvature difference of the adjacent regions to obtain curvature continuity, and performing a weighted combination of the inter-region distance threshold, the normal vector consistency, and the curvature continuity to obtain region similarity; Adjacent regions whose region similarity is greater than a preset merging threshold are dynamically merged, the boundary features and region feature parameters of the merged region are updated, and the merging operation is repeated until the region similarity is less than the preset merging threshold to obtain a candidate point cloud sub-region.

7. The method according to claim 1, characterized in that Extract the position offset and attitude change of the candidate point cloud sub-region in adjacent multi-frame data, construct the motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier, and obtain the traffic sign classification results including: Obtain the spatial coordinates of the candidate point cloud subregion in adjacent multi-frame point cloud data, calculate the coordinate difference between adjacent frames to obtain a position offset matrix, use the coordinate axis direction of the local coordinate system as the main axis direction, calculate the Euler angle change of the candidate point cloud subregion between adjacent frames to obtain a posture change matrix; The projection component of the position offset matrix in three-dimensional space and the pitch angle, roll angle, and yaw angle changes of the attitude change matrix are combined into a six-dimensional vector, a motion state descriptor based on a sliding time window is constructed, principal component analysis is performed on the temporal change characteristics of the motion state descriptor to extract the change pattern of the key dimension, and the change patterns in multiple time windows are cascaded to obtain a target feature vector; The target feature vector is input into a pre-trained support vector classifier to obtain a traffic sign classification result.

8. An image-based 3D point cloud traffic sign classification system, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to collect continuous multi-frame multi-angle image data of the traffic sign to be classified and obtain initial point cloud data based on camera parameters; The second unit is configured to calculate the reflection intensity value and local geometric characteristic value of each point in the initial point cloud data, construct a reflection intensity histogram and a geometric characteristic histogram, and determine an optimal segmentation threshold. The initial point cloud data is divided into a plurality of initial sub-regions according to the optimal segmentation threshold, and the scattering coefficient and absorption coefficient of each initial sub-region are calculated. Adjacent regions are merged based on spatial distribution characteristics to obtain point cloud sub-regions with uniform material. The third unit is used to calculate the local curvature entropy and normal vector entropy of the point cloud sub-region, use the entropy as a weight coefficient to sort the importance, determine the core region and establish a local coordinate system based on the core region, determine the region growth range based on the spatial position relationship and geometric structure relationship, select and dynamically merge the regions that meet the pre-set constraints to obtain candidate point cloud sub-regions; The fourth unit is used to extract the position offset and posture change of the candidate point cloud sub-region in adjacent multi-frame data, construct a motion state descriptor and perform time series association, obtain the target feature vector and input it into the classifier to obtain the traffic sign classification result.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images