A target detection method of point cloud density adaptive integration

Abstract

The present application relates to a kind of target detection methods of point cloud density adaptive integration, belong to autonomous unmanned system environment perception technical field.The method modeling stage is based on three-dimensional columnar voxel division, the flux of beam and normalization density coefficient of voxel are calculated by laser radar parameter, and radar density model is established;While preset Kalman filtering parameter and correction association rule, construct kinematic correction model.Operation stage is first based on radar density model and carries out area screening, and adaptive integration is carried out to effective area, generates global integration point cloud;Second, feature extraction and target detection decoding are carried out to global integration point cloud by coding-decoding network, and output original detection result;Finally, target trajectory is established by matching continuous multiple frames detection result, and velocity is calculated using kinematic correction model and the shape of detection result is corrected.The present application enhances radar point cloud by adaptive integration, and improves the geometric accuracy of target by detection correction.

Description

Technical Field

[0001] This invention belongs to the field of environmental perception technology for autonomous unmanned systems, and relates to a target detection method based on adaptive integration of point cloud density. Background Technology

[0002] LiDAR, as a high-precision environmental perception sensor, is widely used in fields such as autonomous driving and drone surveillance. However, the point cloud data collected by LiDAR in long-distance scenarios is sparse, especially in areas close to the radar's detection limit. The point cloud often only has one or two layers of laser beams projected onto the target surface, resulting in a severe lack of target information and a significant decrease in the performance of target detection algorithms.

[0003] Chinese patent application CN202111191668.5 discloses a laser radar point cloud integration target detection method. This method constructs a point cloud time integration model for non-terrestrial point clouds, performs different time integration operations based on point cloud distances, then clusters the integrated point clouds and applies tracking filtering to the clustered targets to obtain their motion states. However, this method only uses point cloud integration as a preprocessing step, which is relatively independent of the target detection model and cannot optimize the integration strategy for the detection task. Chinese patent application CN202310978208.X discloses a multi-sensor fusion target detection method, but this method mainly focuses on mutual heuristic learning between different sensors and does not address the point cloud sparsity problem of a single laser radar in long-distance scenarios. Chinese patent application: A method for accurately estimating the geometric features of a target by integrating point clouds (application number: CN202411472902.5) discloses a method for accurately estimating geometric features after integrating point clouds. This method constructs system state equations and measurement equations and uses Kalman filtering for analytical optimal estimation. However, it fails to integrate the integration enhancement strategy with the target detection network into an integrated design, making it difficult to systematically improve target detection performance from an end-to-end perspective.

[0004] Overall, current lidar solutions still suffer from issues such as missing target information and reduced detection performance due to sparse point clouds in long-distance scenarios. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a target detection method with adaptive integration of point cloud density, which integrates adaptive integration and detection correction into the target detection network, so as to optimize the point cloud enhancement strategy and the detection task in a coordinated manner, realize end-to-end sparse point cloud target detection, and effectively improve the performance of long-distance target detection.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A target detection method based on adaptive integration of point cloud density is disclosed. This method includes a modeling phase and an execution phase. In the modeling phase, a lidar density model and a kinematic correction model are constructed. In the execution phase, adaptive integration of point cloud density, feature extraction and detection, and kinematic detection correction are performed to obtain the final target detection result. The radar density model is based on the division of three-dimensional spatial columnar voxels. The laser emission direction is calculated according to the resolution parameters of the lidar, and the density coefficient is obtained according to the beam flux of each columnar voxel. The motion correction model provides parameter and rule support for subsequent velocity estimation and detection box position offset and volume expansion correction based on preset Kalman filter parameters, velocity classification threshold, velocity-correction coefficient mapping relationship and smoothing verification parameters. The point cloud adaptive integration filters the effective integration region based on the beam flux of adjacent frames, then calculates the number of point clouds in each voxel and multiplies it with the corresponding normalized density coefficient in the radar density model to obtain the normalized density of the voxel. Based on the normalized density of the effective region at close range, the integration times of each effective region are determined by the density ratio. After iteration, the integration and stitching of point clouds in multiple frames are completed to generate a global integrated point cloud. Feature extraction and detection take the global integral point cloud as input. First, the encoding network performs feature encoding and convolution operations to generate a depth feature map. Then, the decoding network performs decoding prediction on the depth feature map. After completing the target detection, the original detection result is output. Kinematic detection correction is based on the detection results of multiple consecutive frames. The target trajectory is established by matching the detection boxes of adjacent frames through cross-union ratio. The velocity is estimated by using Kalman filtering in the kinematic correction parameter model. Then, the correction rules and parameters in the model are combined to correct the positional offset and volume expansion of the detection results, and finally output accurate detection results.

[0007] Furthermore, the radar density model establishes a cylindrical voxel grid with the radar as the coordinate origin, and the grid is defined as follows: ,in Represents a single columnar voxel. 、 For this voxel in The coordinate position along the axis. , They represent Boundary range in the axial direction; The vertical detection range corresponding to the axial direction is set to ; columnar serotonin in The side length of the plane is The horizontal resolution of the lidar is Vertical resolution is ; Next, the beam distribution pattern is set, including vertical resolution and horizontal resolution. Vertical resolution represents the perpendicular angle of each beam to the center line of the lidar, which is a set of different angles. Horizontal resolution The horizontal angle representing each increment is:

[0008] in, k The index of the horizontal angle, with a value range of 1. ; Next, calculate the beam direction:

[0009] Then, the theoretical density is calculated, assuming the origin of the lidar coordinate system is at a point outside space. And the direction of the light is determined by the vector Given, any point on the ray... Represented as:

[0010] in ， is the side length of the voxel mesh; For each ray, starting from the lidar position, calculate the voxels it passes through sequentially along the ray direction, and record the number of times each voxel is traversed. Voxel index. The calculation formula is:

[0011] The voxel grid beam flux is:

[0012] in, This is an indicator function that, when the condition is true, The value is 1 if it is 1, otherwise it is 0. This is the density normalization coefficient.

[0013] Furthermore, the kinematic correction parameter model provides support for velocity estimation and detection box correction during the operational phase by pre-setting the parameters required for velocity estimation, establishing a quantitative mapping relationship between velocity and correction coefficients, and setting time-series smoothing verification rules. The process includes: A uniform Kalman filter model is used with pre-defined kinematic estimation parameters. The state vector, state transition matrix, observation matrix, and noise covariance matrix are defined, respectively. The state vector is defined as:

[0014] in , For the goal The center coordinates of the plane , For the goal Velocity in the axial direction; The state transition matrix is ​​set as follows:

[0015] in The time interval between adjacent frames; The observation matrix is ​​defined as:

[0016] The process noise covariance matrix Q is defined as:

[0017] in , This is the process noise covariance parameter; The observation noise covariance matrix is ​​defined as:

[0018] in To observe the noise covariance parameter; Establish a speed classification threshold calibration. The speed classification threshold is preset based on the characteristics of the LiDAR application scenario and the typical target motion law. The classification includes at least low speed, medium speed and high speed levels. The setting is based on the physical characteristics of the target motion and the requirements of the application scenario: combined with the typical motion speed range of different types of targets within the LiDAR coverage area, the speed range of each level is divided. At the same time, it adapts to the safety requirements of the scenario to ensure that the high speed level can cover the maximum target speed that may occur in the scenario, and the low speed level can accurately match stationary or slowly moving targets. A quantization mapping relationship between speed and correction coefficients is established. This mapping is achieved through a fully connected network, and the mapping function is... , To maintain a shared network structure, only the output layer parameters are independent; the input layer of a fully connected network consists of one-dimensional neurons, i.e., velocity levels. The encoded values ​​are set; the hidden layers are set with multi-layer activation functions of ReLU and LeakyReLU; the output layer is a one-dimensional neuron, using the Sigmoid activation function, and the correction coefficient is constrained to a reasonable range of 0~1; the network uses the Adam optimizer, and the learning rate adopts an adaptive decay strategy. Preset timing smoothing and rationality verification parameters, including position offset smoothing window. Volume reference window Speed ​​trend adaptation threshold Volume change rate threshold Conservative correction coefficient .

[0019] Furthermore, training is conducted on the quantization mapping relationship between velocity and correction coefficients. This involves using continuous target trajectories from the training set as a basis, selecting multiple consecutive frames of data for each trajectory segment, and calculating the velocity level for each frame. At the same time, the actual position offset of the detection box in that frame is marked. Actual volume ;Will As input, , As respectively , The supervisory signal is used to construct training sample pairs to ensure that the number of training samples meets the model convergence requirements. Design the total loss function:

[0020] in, The initial offset between the original detection box and the predicted trajectory is obtained by the difference between the Kalman filter prior estimate and the observed value. This is the average volume of the original detection bounding boxes across multiple consecutive frames; This is a monotonic constraint loss, used to constrain "the higher the speed level, the more..." The larger, The smaller the size, the better; For constraint weights.

[0021] Furthermore, the adaptive integral, based on the density judgment results of the radar density model, achieves accurate fusion of multi-frame point clouds through effective region selection and dynamic adjustment of the number of integrations. The process includes: The effective region is determined by the consistency of beam flux in adjacent frames: compare with the current frame. With the next frame Beam flux of a single voxel and If both are 0, the region is marked as invalid and will not participate in the integration; if either is not 0, the region is marked as valid and will proceed to the integration process. Set the integration time window size As the maximum number of integration attempts, the real-time collected point cloud data is mapped to the pre-defined columnar voxel mesh during the modeling stage, and the number of point clouds within each voxel is counted. Density normalization coefficients obtained by combining radar density models Calculate the normalized density of each voxel. The expression is:

[0022] in This ensures the effectiveness of the normalized density; Then, set the close-range detection threshold. Select the initial integration frame whose distance from the radar origin does not exceed The effective region is determined, and the mean of the normalized density within that region is calculated as the baseline density. Set the integral number adjustment coefficient Calculate the ratio of the normalized density to the baseline density for each effective region:

[0023] Based on ratio Determine the target integration number for each effective region. :

[0024] constraint The range of values ​​is ; Next, using the initial integration frame Starting with the current frame, subsequent point clouds are selected sequentially as frames to be integrated. The point clouds of the corresponding valid regions of the current frame and the frames to be integrated are merged. After merging each frame, the normalized density of the region is recalculated, and it is determined whether the integration termination condition is met: the cumulative number of integrations reaches the target number of integrations. Alternatively, the recalculated normalized density may reach the baseline density; if either condition is met, integration in that region ceases; otherwise, integration continues in the next frame until all valid regions meet the termination condition or the cumulative integration count reaches the threshold. ; Finally, collect the point cloud data of all valid regions after iterative merging, and sort them according to each region. Coordinate index of the plane Spatial stitching is performed to restore the complete three-dimensional point cloud spatial structure and generate a global integral point cloud, providing rich point cloud information support for subsequent feature extraction and detection.

[0025] Furthermore, an architecture of encoder-network feature extraction and decoder-network detection / decoding is adopted to perform deep feature extraction and target parameter prediction on the global integral point cloud. The encoder-network feature extraction process is as follows: Feature encoding is performed on each voxel of the global integral point cloud to extract the raw features, which include: the three-dimensional coordinates of the point. Laser reflection intensity, average coordinates of point clouds within voxels The offset of the point coordinates relative to the midpoint of the voxel ; The original features of each voxel are input into the encoding network, and a high-dimensional feature vector with fixed dimensions is output through a multilayer perceptron. Encode feature maps The input is fed into the backbone network, and convolutional operations are used to enhance the receptive field of the feature maps, thereby fusing global semantic features with local detail features and outputting deep feature maps. Complete the feature extraction process of the coding network.

[0026] Furthermore, the decoding process of the decoding network detection is as follows: Using a detection head to analyze deep feature maps To perform prediction, the detection head includes a classification branch and a regression branch: the classification branch outputs the target class confidence score for each grid position, expressed as:

[0027] The regression branch outputs the core parameters of the target, including the coordinates of the detection box center, the bounding box size, and the rotation angle, expressed as follows:

[0028] Based on multiple pre-set anchor box parameters adapted to different target types, the parameters output by the regression branch are combined with the anchor boxes, and the original detection box set for each frame is obtained through decoding. This completes the detection and decoding process of the decoding network.

[0029] Furthermore, based on continuous frame detection results and kinematic correction parameter models, precise optimization of the detection box position and volume is achieved through trajectory establishment, velocity estimation, and temporal smoothing correction. This includes establishing a target trajectory based on the intersection-over-union ratio (IoU), performing uniform Kalman filtering velocity estimation, and a temporal smoothing shape correction process. The process of establishing the target trajectory based on IoU is as follows: First, obtain continuous The original detection box set of the frame Set confidence threshold Filter out The low-confidence detection boxes are used to obtain the preprocessed set of detection boxes. };Calculate the first Frame detection box With the Frame detection box Intersection over Union (IOU) measures the degree of overlap between two bounding boxes.

[0030] In the formula, Indicates the volume of the detection frame. This represents the intersection region of the two detection frames. The region is the union of regions; Set IOU matching threshold IOU th If the two detection boxes If the target categories are consistent, then an association is established between the two, and they are determined to be the same target. For consecutive... The above matching operation is repeated for each frame to form a continuously associated target trajectory: .

[0031] Furthermore, by invoking the preset Kalman filter parameters in the kinematic correction parameter model, and based on the continuous detection box data of the target trajectory, velocity estimation is completed through an iterative prediction-update process. The process is as follows: Prediction phase: based on the first Posterior state estimation of frames and state transition matrix Calculate the first Prior state estimation of frames and prior error covariance matrix :

[0032]

[0033] In the formula, For the first The posterior error covariance matrix of the frame. This is the transpose of the state transition matrix. The process noise covariance matrix; Update phase: based on the first The center coordinates of the frame detection box are the observed values. Calculate Kalman gain :

[0034] In the formula, For the observation matrix, For the transpose of the observation matrix, To observe the noise covariance matrix, Perform matrix inversion; correct the prior estimate based on Kalman gain to obtain the first... Posterior state estimation of frames and posterior error covariance matrix :

[0035]

[0036] In the formula, For the identity matrix, from Extracting velocity components As the first The velocity estimate of the target in the frame; Used to characterize the confidence level of the estimation results.

[0037] Furthermore, by combining the velocity estimate with multi-frame temporal information, position offset correction and volume expansion suppression are performed respectively. The processes include: For position offset correction, the theoretical position of the target is first predicted based on the velocity estimate output by the Kalman filter, and the initial position offset is obtained by subtracting the original detection box center coordinates from this estimate. , Introducing the posterior error covariance matrix Calculate the confidence weighting coefficient :

[0038] In the formula, This is a matrix trace operation, where C is a preset calibration constant. The initial offset is corrected using weighted coefficients.

[0039] For continuous The weighted offsets of the frames are averaged to obtain the time-smoothed offsets. , To reduce noise interference in a single frame, calculate the velocity change between adjacent frames. ,like Then, the smoothed offset is adjusted for trend adaptation, and finally the adjusted offset is superimposed on the original detection box center coordinates to obtain the corrected center coordinates.

[0040]

[0041] In the formula, , The coordinates of the original detection box center are: , This is the offset after trend adaptation.

[0042] For volume expansion suppression, the target velocity modulus is first calculated. Speed ​​levels are determined based on speed grading thresholds from a kinematically corrected parameter model. Call the mapping function Matching yields volume correction coefficients ; in continuous The average volume of the original detection bounding box in the frame is used as the temporal volume benchmark. Calculate the initial correction volume:

[0043] Calculate the initial corrected volume and the original volume of the current frame. volume change rate Verify the rationality of the correction.

[0044] like Then a conservative correction factor is used. Recalculate: ;like Then, temporal smoothing is performed on the corrected volume of multiple consecutive frames to obtain the final corrected volume. ; By fusing the corrected center coordinates and volume information, an accurate final detection bounding box is output.

[0045] The beneficial effects of this invention are as follows: This invention proposes a target detection method based on adaptive integration of point cloud density. Based on density judgment and correction rules and parameters provided during the modeling stage, it integrates adaptive integration, feature extraction and detection, and kinematic detection correction into a single design during the runtime phase. Its effects include at least the following: Integral Enhancement of Point Cloud Information: Based on the radar density model, this method accurately identifies sparse point cloud regions at long distances through beam flux calculation and normalized density quantization. Using the normalized density of the effective near-range region as a benchmark, target information is supplemented by accumulating point clouds from multiple frames, effectively addressing the problem of missing target information in long-range scenarios.

[0046] Adaptive mechanisms reduce redundant operations: An effective region screening step is added before integration. By comparing the beam flux of the same columnar voxel in adjacent frames, invalid regions where no laser beam passes through are excluded, thus avoiding unnecessary resource occupation. At the same time, the number of integrations is dynamically adjusted based on the ratio of normalized density to reference density. Fewer integrations are set for high-density point cloud regions in close proximity, thus avoiding information redundancy and computational waste caused by over-integration.

[0047] Kinematic detection correction improves accuracy: To address the detection box position offset and volume expansion caused by the integration process, a continuous target trajectory is established through cross-union ratio matching, and the target velocity is estimated using Kalman filtering; based on the correction coefficient of velocity level mapping adaptation, combined with confidence weighting, temporal smoothing constraints and volume change rate verification, the detection box position and volume are corrected to offset the deviation caused by integration.

[0048] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1This is a schematic diagram of the overall process of a point cloud density adaptive integration target detection method according to an embodiment of the present invention; Figure 2 This is a logical schematic diagram of the point cloud adaptive integration strategy according to an embodiment of the present invention; Figure 3 This is a top view of the radar density model according to an embodiment of the present invention. Detailed Implementation

[0050] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0051] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0052] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0053] Please see Figures 1-3 This is a target detection method based on adaptive integration of point cloud density.

[0054] Example This embodiment describes in detail a target detection method based on adaptive integration of point cloud density, which includes a modeling stage and an operation stage. In the modeling stage, a radar density model and a kinematic correction parameter model are established to provide density judgment basis and preset rules and parameter support for the operation stage. In the operation stage, based on the model built in the modeling stage, adaptive integration of point cloud, feature extraction and detection, and kinematic estimation correction are completed, and finally accurate detection results are output. Figure 1 This paper presents an overall flowchart of a point cloud density adaptive integration target detection method according to the present invention. The flowchart includes the following steps: (1) Radar density model A cylindrical voxel mesh is established using the radar as the coordinate origin. The mesh is defined as follows: ,in Represents a single columnar voxel. 、 Let be the coordinates of the voxel in the x and y directions. , They represent x, y The boundary range in the z-axis direction; the vertical detection range corresponding to the z-axis direction, set as follows. The side length of the columnar voxel in the xoy plane is v; the horizontal resolution of the lidar is... Vertical resolution is This allows for the calculation of the ray direction of the lidar, thus representing lidar with arbitrary beams. Centered on the origin of the lidar coordinate system, the point cloud data is voxelized, and the density normalization coefficient of each voxel mesh is calculated using ray tracing methods. .

[0055] (1.1) Beam distribution mode setting The vertical resolution is a set of different angles, representing the perpendicular angle of each line beam to the center line of the lidar: Horizontal resolution This represents the horizontal angle for each increment. ,in, k The index of the horizontal angle, with a value range of 1. .

[0056] (1.2) Calculation of beam direction:

[0057] in .Right now:

[0058] The above method can be used to represent lidar with arbitrary line beams and resolutions.

[0059] (1.3) Calculation of theoretical density Assume the origin of the lidar coordinate system is at a point outside space. And the direction of the light is determined by the vector Given: any point on the ray. It can be represented as:

[0060] in ， is the side length of the voxel mesh.

[0061] For each ray, starting from the lidar position, calculate the voxels it passes through sequentially along the ray direction, and record the number of times each voxel is traversed. Voxel Index The calculation formula is

[0062] Voxel grid beam flux is

[0063] in, This is an indicator function that, when the condition is true, The value is 1 if it is 1, otherwise it is 0. This is the density normalization coefficient.

[0064] (2) Kinematic correction parameter model The system pre-defines the parameters required for velocity estimation, establishes a quantitative mapping relationship between velocity and correction coefficients, and sets temporal smoothing verification rules to support velocity estimation and detection box correction during the runtime phase. Its construction process requires calibration using historical labeled data to ensure that the parameters are adapted to the actual target motion characteristics in the scene.

[0065] (2.1) Preset of kinematic estimation parameters A uniform Kalman filter model is adopted, and its core parameters include the state vector, state transition matrix, observation matrix, and noise covariance matrix. The state vector is defined as follows:

[0066] in , For the goal The center coordinates of the plane , For the goal Velocity in the axial direction; The state transition matrix is ​​set as follows:

[0067] in The time interval between adjacent frames; The observation matrix is ​​set as follows:

[0068] The process noise covariance matrix Q is set as follows:

[0069] in , This is the process noise covariance parameter; The observation noise covariance matrix is ​​set as follows:

[0070] in To observe the noise covariance parameter.

[0071] (2.2) Velocity grading and mapping function training method.

[0072] Speed ​​grading threshold calibration: The speed grading threshold is preset based on the characteristics of the LiDAR application scenario and the typical target motion patterns. The number of grading levels is no less than 3, that is, it includes at least low speed, medium speed, and high speed levels. The setting is based on the physical characteristics of target motion and application scenario requirements: combined with the typical motion speed range of different types of targets within the LiDAR coverage area, the speed range of each level is divided; at the same time, it adapts to the scenario safety requirements, ensuring that the high speed level can cover the maximum target speed that may occur in the scenario, and the low speed level can accurately match stationary or slowly moving targets.

[0073] Mapping function training method: The quantization mapping between speed level and correction coefficient is achieved through a fully connected network. , The network structure can be shared, with only the output layer parameters remaining independent. The specific design is as follows. Network structure: The input layer consists of one-dimensional neurons, i.e., speed levels. The encoded values ​​are set; the hidden layers are set with multi-layer activation functions of ReLU and LeakyReLU; the output layer is a one-dimensional neuron, using the Sigmoid activation function, and the correction coefficient is constrained to a reasonable range of 0~1; the network uses the Adam optimizer, and the learning rate adopts an adaptive decay strategy.

[0074] Training data construction: Based on the continuous target trajectories in the training set, multiple consecutive frames of data are selected for each trajectory segment, and the velocity level of each frame is calculated. At the same time, the actual position offset of the detection box in that frame is marked. Actual volume ;Will As input, , As respectively , The supervisory signal is used to construct training sample pairs to ensure that the number of training samples meets the model convergence requirements.

[0075] Loss function and training process: Total loss function The calculation details for each part are as follows: The initial offset between the original detection box and the predicted trajectory is obtained by the difference between the Kalman filter prior estimate and the observed value. This is the average volume of the original detection bounding boxes across multiple consecutive frames; This is a monotonic constraint loss, used to constrain "the higher the speed level, the more..." The larger, The smaller the size, the better; For constraint weights.

[0076] (2.3) Smoothing and verification parameter settings and dataset adaptation Preset timing smoothing and rationality verification parameters, with core parameters including the position offset smoothing window. Volume reference window Speed ​​trend adaptation threshold Volume change rate threshold Conservative correction coefficient The settings for each parameter need to be combined with the target motion characteristics and lidar parameters in the calibration dataset. The specific setting logic is as follows: Window parameter settings: , The correction is related to the lidar frame rate and the target speed level, and is determined statistically based on the calibration dataset. High-speed targets move rapidly, so a larger smoothing window is set to improve correction stability. Low-speed targets move smoothly, so a smaller window can be set to ensure response speed, thus ensuring that the correction results for targets of different speeds can balance response speed and stability.

[0077] Establish the association rules between speed level and the above parameters, and achieve dynamic matching through parameter lookup table. This completes the construction of the kinematic correction parameter model.

[0078] (3) Adaptive Integral Adaptive integration, based on density judgment results from the radar density model, achieves accurate fusion of multi-frame point clouds through effective region filtering and dynamic adjustment of the integration count, avoiding invalid calculations and information redundancy. The logical flow is as follows: Figure 2 As shown; (3.1) Valid region selection and normalized density calculation The effective region is determined by the consistency of beam flux in adjacent frames: compare with the current frame. With the next frame Beam flux of a single voxel and If both are 0, the region is marked as invalid and will not participate in the integration; if either is not 0, the region is marked as valid and will proceed to the integration process.

[0079] Set the integration time window size As the maximum number of integration attempts, the real-time collected point cloud data is mapped to the pre-defined columnar voxel mesh during the modeling stage, and the number of point clouds within each voxel is counted. The density normalization coefficient obtained by combining the radar density model Calculate the normalized density of each voxel. The expression is:

[0080] in This ensures the effectiveness of the normalized density.

[0081] (3.2) Calculation of baseline density and target integral degree Set a close-range detection threshold Select the initial integration frame whose distance from the radar origin does not exceed The effective region is determined, and the mean of the normalized density within that region is calculated as the baseline density. Set the integral number adjustment coefficient. Calculate the ratio of the normalized density to the baseline density for each effective region, expressed as follows:

[0082] Based on ratio Determine the target integration number for each effective region. The calculation expression is:

[0083] constraint The range of values ​​is To avoid having too few or too many points.

[0084] (3.3) Iterative integration implementation With the initial integration frame Starting with the current frame, subsequent point clouds are selected sequentially as frames to be integrated. The point clouds of the corresponding valid regions of the current frame and the frames to be integrated are merged. After merging each frame, the normalized density of the region is recalculated, and it is determined whether the integration termination condition is met: the cumulative integration count reaches the target integration count. Alternatively, the recalculated normalized density may reach the baseline density. If either condition is met, integration in that region ceases; otherwise, integration continues in the next frame until all valid regions meet the termination condition or the cumulative integration count reaches the threshold. .

[0085] (3.4) Global Integral Point Cloud Stitching Collect point cloud data of all valid regions after iterative merging, and index each region according to its coordinates in the xoy plane. Spatial stitching is performed to restore the complete three-dimensional point cloud spatial structure and generate a global integral point cloud, providing rich point cloud information support for subsequent feature extraction and detection.

[0086] (4) Feature extraction and detection An architecture of encoding network feature extraction and decoding network detection is adopted to perform deep feature extraction and target parameter prediction on the global integral point cloud. The specific implementation steps are as follows: (4.1) Feature extraction of coding network Feature encoding is performed on each voxel of the global integral point cloud, and the extracted raw features include: the three-dimensional coordinates of the point. Laser reflection intensity, average coordinates of point clouds within voxels The offset of the point coordinates relative to the midpoint of the voxel The raw features of each voxel are input into the encoding network, and a high-dimensional feature vector of fixed dimensions is output through a multilayer perceptron. This is based on columnar voxels. Coordinate Index The high-dimensional feature vectors of all voxels are mapped onto a two-dimensional grid to form an encoded feature map. ,in , These represent the height and width of the feature map, respectively, corresponding to the columnar voxel grid in... The number of axes is adjusted to accommodate subsequent convolutional processing. This will encode the feature map. The input is fed into the backbone network, and convolutional operations are used to enhance the receptive field of the feature maps, thereby fusing global semantic features with local detail features and outputting deep feature maps. Complete the feature extraction process of the coding network.

[0087] (4.2) Decoding Network Detection Decoding Using a detection head to analyze deep feature maps To perform prediction, the detection head includes a classification branch and a regression branch: the classification branch outputs the target class confidence score for each grid position, expressed as:

[0088] The regression branch outputs the core parameters of the target, including the coordinates of the detection box center, the bounding box size, and the rotation angle, expressed as follows:

[0089] Based on multiple pre-set anchor box parameters adapted to different target types, the parameters output by the regression branch are combined with the anchor boxes, and the original detection box set for each frame is obtained through decoding. This completes the detection and decoding process of the decoding network.

[0090] (5) Kinematic test correction Based on continuous frame detection results and kinematic correction parameter models, the position and volume of the detection box are accurately optimized through trajectory establishment, velocity estimation, and temporal smoothing correction. The specific implementation steps are as follows: (5.1) Establishment of target trajectory based on intersection-union ratio Get continuous The original detection box set of the frame Set confidence threshold Filter out The low-confidence detection boxes are used to obtain the preprocessed set of detection boxes. }. Calculate the first Frame detection box With the Frame detection box The Intersection over Union (IOU) measures the degree of overlap between two bounding boxes, and is calculated using the following formula:

[0091] In the formula, Indicates the volume of the detection frame. This represents the intersection region of the two detection frames. This is the union region. Set the IOU matching threshold (IOU). th If the two detection boxes If the target categories are identical, an association is established between the two, classifying them as the same target. This matching operation is repeated for K consecutive frames to form continuously associated target trajectories.

[0092] (5.2) Uniform Kalman Filter Velocity Estimation By calling the preset Kalman filter parameters in the kinematic correction parameter model, and based on the continuous detection box data of the target trajectory, velocity estimation is completed through an iterative prediction-update process. The specific steps are as follows: Prediction phase: based on the first Posterior state estimation of frames and state transition matrix Calculate the first Prior state estimation of frames and prior error covariance matrix The expression is

[0093]

[0094] In the formula, For the first The posterior error covariance matrix of the frame. This is the transpose of the state transition matrix. Let be the process noise covariance matrix.

[0095] Update phase: based on the first The center coordinates of the frame detection box are the observed values. Calculate Kalman gain The expression is

[0096] In the formula, For the observation matrix, For the transpose of the observation matrix, To observe the noise covariance matrix, This involves matrix inversion. The prior estimate is corrected based on Kalman gain to obtain the... Posterior state estimation of frames and posterior error covariance matrix The expression is

[0097]

[0098] In the formula, It is an identity matrix. From... Extracting velocity components As the first The velocity estimate of the target in the frame; Used to characterize the confidence level of the estimation results.

[0099] (5.3) Temporally smoothing shape correction By combining the velocity estimate with multi-frame temporal information, position offset correction and volume expansion suppression are performed respectively. The specific process is as follows: (5.3.1) Position offset correction The theoretical target position is predicted based on the velocity estimate output by the Kalman filter, and the initial position offset is obtained by subtracting the original detection box center coordinates from the predicted theoretical target position. , Introducing the posterior error covariance matrix. Calculate the confidence weighting coefficient The expression is:

[0100] In the formula, This is a matrix trace operation, where C is a preset calibration constant. The initial offset is corrected using weighted coefficients.

[0101] For continuous The weighted offsets of the frames are averaged to obtain the time-smoothed offsets. , To reduce noise interference in a single frame, calculate the velocity change between adjacent frames. ,like Then, the smoothed offset is adjusted for trend adaptation, and finally the adjusted offset is superimposed on the original detection box center coordinates to obtain the corrected center coordinates.

[0102]

[0103] In the formula, , The coordinates of the original detection box center are: , This is the offset after trend adaptation.

[0104] (5.3.2) Volume expansion correction First, calculate the target velocity magnitude. Speed ​​levels are determined based on speed grading thresholds from a kinematically corrected parameter model. Call the mapping function Matching yields volume correction coefficients In continuous The average volume of the original detection bounding box in the frame is used as the temporal volume benchmark. Calculate the initial correction volume:

[0105] Calculate the initial corrected volume and the original volume of the current frame. volume change rate Verify the rationality of the correction.

[0106] like Then a conservative correction factor is used. Recalculate: ;like Then, temporal smoothing is performed on the corrected volume of multiple consecutive frames to obtain the final corrected volume. By fusing the corrected center coordinates and volume information, a precise final detection bounding box is output.

[0107] Figure 3 This is a top view of the radar density model of the present invention, with the lidar as the center and the map as the center. xoy In the planar projection map, under this map model, the beam flux passing through each grid is calculated using the beam direction calculation formula and ray tracing method. The normalized density coefficients are obtained. In adaptive integration, voxels marked in black with zero flux are not included in subsequent iterations to reduce redundancy and improve computational performance.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A target detection method based on adaptive integration of point cloud density, characterized in that: The method includes a modeling phase and an execution phase. In the modeling phase, a lidar density model and a kinematic correction model are constructed. In the execution phase, point cloud adaptive integration, feature extraction and detection, and kinematic detection correction are performed to obtain the final target detection result. The radar density model is based on the division of three-dimensional spatial columnar voxels. The laser emission direction is calculated according to the resolution parameters of the lidar, and the density coefficient is obtained according to the beam flux of each columnar voxel. The motion correction model provides parameter and rule support for subsequent velocity estimation and detection box position offset and volume expansion correction based on preset Kalman filter parameters, velocity classification threshold, velocity-correction coefficient mapping relationship and smoothing verification parameters. The point cloud adaptive integration filters the effective integration region based on the beam flux of adjacent frames, then calculates the number of point clouds in each voxel and multiplies it with the corresponding normalized density coefficient in the radar density model to obtain the normalized density of the voxel. Based on the normalized density of the effective region at close range, the integration times of each effective region are determined by the density ratio. After iteration, the integration and stitching of point clouds in multiple frames are completed to generate a global integrated point cloud. Feature extraction and detection take the global integral point cloud as input. First, the encoding network performs feature encoding and convolution operations to generate a depth feature map. Then, the decoding network performs decoding prediction on the depth feature map. After completing the target detection, the original detection result is output. Kinematic detection correction is based on the detection results of multiple consecutive frames. The target trajectory is established by matching the detection boxes of adjacent frames through cross-union ratio. The velocity is estimated by using Kalman filtering in the kinematic correction parameter model. Then, the correction rules and parameters in the model are combined to correct the positional offset and volume expansion of the detection results, and finally output accurate detection results.

2. The target detection method based on adaptive integration of point cloud density according to claim 1, characterized in that: The radar density model establishes a cylindrical voxel mesh with the radar as the coordinate origin. The mesh is defined as follows: ,in Represents a single columnar voxel. 、 For this voxel in The coordinate position along the axis. , They represent Boundary range in the axial direction; The vertical detection range corresponding to the axial direction is set to ; columnar serotonin in The side length of the plane is The horizontal resolution of the lidar is Vertical resolution is ; Next, the beam distribution pattern is set, including vertical resolution and horizontal resolution. Vertical resolution represents the perpendicular angle of each beam to the center line of the lidar, which is a set of different angles. Horizontal resolution The horizontal angle representing each increment is: in, k The index of the horizontal angle, with a value range of 1. ; Next, calculate the beam direction: Then, the theoretical density is calculated, assuming the origin of the lidar coordinate system is at a point outside space. And the direction of the light is determined by the vector Given, any point on the ray... Represented as: in ， is the side length of the voxel mesh; For each ray, starting from the lidar position, calculate the voxels it passes through sequentially along the ray direction, and record the number of times each voxel is traversed. Voxel index. The calculation formula is: The voxel grid beam flux is: in, This is an indicator function that, when the condition is true, The value is 1 if it is not 1, otherwise it is 0. This is the density normalization coefficient.

3. The target detection method based on adaptive integration of point cloud density according to claim 2, characterized in that: The kinematic correction parameter model provides support for velocity estimation and detection box correction during the operational phase by pre-setting the parameters required for velocity estimation, establishing a quantitative mapping relationship between velocity and correction coefficients, and setting time-series smoothing verification rules. The process includes: A uniform Kalman filter model is used with pre-defined kinematic estimation parameters. The state vector, state transition matrix, observation matrix, and noise covariance matrix are defined, respectively. The state vector is defined as: in , For the goal The center coordinates of the plane , For the goal Velocity in the axial direction; The state transition matrix is ​​set as follows: in The time interval between adjacent frames; The observation matrix is ​​defined as: The process noise covariance matrix Q is defined as: in , This is the process noise covariance parameter; The observation noise covariance matrix is ​​defined as: in To observe the noise covariance parameter; Establish a speed classification threshold calibration. The speed classification threshold is preset based on the characteristics of the LiDAR application scenario and the typical target motion law. The classification includes at least low speed, medium speed and high speed levels. The setting is based on the physical characteristics of the target motion and the requirements of the application scenario: combined with the typical motion speed range of different types of targets within the LiDAR coverage area, the speed range of each level is divided. At the same time, it adapts to the safety requirements of the scenario to ensure that the high speed level can cover the maximum target speed that may occur in the scenario, and the low speed level can accurately match stationary or slowly moving targets. A quantization mapping relationship between speed and correction coefficients is established. This mapping is achieved through a fully connected network, and the mapping function is... , To maintain a shared network structure, only the output layer parameters are independent; the input layer of a fully connected network consists of one-dimensional neurons, i.e., velocity levels. The encoded values ​​are set; the hidden layers are set with multi-layer activation functions of ReLU and LeakyReLU; the output layer is a one-dimensional neuron, using the Sigmoid activation function, and the correction coefficient is constrained to a reasonable range of 0~1; the network uses the Adam optimizer, and the learning rate adopts an adaptive decay strategy. Preset timing smoothing and rationality verification parameters, including position offset smoothing window. Volume reference window Speed ​​trend adaptation threshold Volume change rate threshold Conservative correction coefficient .

4. The target detection method based on adaptive integration of point cloud density according to claim 3, characterized in that: Training is performed to establish the quantization mapping relationship between velocity and correction coefficient. This involves using continuous target trajectories from the training set as a basis, selecting multiple consecutive frames of data for each trajectory segment, and calculating the velocity level for each frame. At the same time, the actual position offset of the detection box in that frame is marked. Actual volume ;Will As input, , As respectively , The supervisory signal is used to construct training sample pairs to ensure that the number of training samples meets the model convergence requirements. Design the total loss function: in, The initial offset between the original detection box and the predicted trajectory is obtained by the difference between the Kalman filter prior estimate and the observed value. This is the average volume of the original detection bounding boxes across multiple consecutive frames; This is a monotonic constraint loss, used to constrain "the higher the speed level, the more..." The larger, The smaller the size, the better; For constraint weights.

5. The target detection method based on adaptive integration of point cloud density according to claim 3, characterized in that: Adaptive integration, based on the density judgment results of the radar density model, achieves accurate fusion of multi-frame point clouds through effective region selection and dynamic adjustment of the integration count. The process includes: The effective region is determined by the consistency of beam flux in adjacent frames: compare with the current frame. With the next frame Beam flux of a single voxel and If both are 0, the region is marked as invalid and will not participate in the integration; if either is not 0, the region is marked as valid and will proceed to the integration process. Set the integration time window size As the maximum number of integration attempts, the real-time collected point cloud data is mapped to the pre-defined columnar voxel mesh during the modeling stage, and the number of point clouds within each voxel is counted. Density normalization coefficients obtained by combining radar density models Calculate the normalized density of each voxel. The expression is: in This ensures the effectiveness of the normalized density; Then, set the close-range detection threshold. Select the initial integration frame whose distance from the radar origin does not exceed The effective region is determined, and the mean of the normalized density within that region is calculated as the baseline density. Set the integral number adjustment coefficient Calculate the ratio of the normalized density to the baseline density for each effective region: Based on ratio Determine the target integration number for each effective region. : constraint The range of values ​​is ; Next, using the initial integration frame Starting with the current frame, subsequent point clouds are selected sequentially as frames to be integrated. The point clouds of the corresponding valid regions of the current frame and the frames to be integrated are merged. After merging each frame, the normalized density of the region is recalculated, and it is determined whether the integration termination condition is met: the cumulative number of integrations reaches the target number of integrations. Alternatively, the recalculated normalized density may reach the baseline density; if either condition is met, integration in that region ceases; otherwise, integration continues in the next frame until all valid regions meet the termination condition or the cumulative integration count reaches the threshold. ; Finally, collect the point cloud data of all valid regions after iterative merging, and index each region according to its coordinates in the xoy plane. Spatial stitching is performed to restore the complete three-dimensional point cloud spatial structure and generate a global integral point cloud, providing rich point cloud information support for subsequent feature extraction and detection.

6. The target detection method based on adaptive integration of point cloud density according to claim 5, characterized in that: An architecture of encoder-decoder network feature extraction and decoder network detection is adopted to perform deep feature extraction and target parameter prediction on the global integral point cloud. The feature extraction process of the encoder-decoder network is as follows: Feature encoding is performed on each voxel of the global integral point cloud to extract the raw features, which include: the three-dimensional coordinates of the point. Laser reflection intensity, average coordinates of point clouds within voxels The offset of the point coordinates relative to the midpoint of the voxel ; The original features of each voxel are input into the encoding network, and a high-dimensional feature vector with fixed dimensions is output through a multilayer perceptron. Encode feature maps The input is fed into the backbone network, and convolutional operations are used to enhance the receptive field of the feature maps, thereby fusing global semantic features with local detail features and outputting deep feature maps. Complete the feature extraction process of the coding network.

7. The target detection method based on adaptive integration of point cloud density according to claim 6, characterized in that: The decoding process of the network detection is as follows: Using a detection head to analyze deep feature maps To perform prediction, the detection head includes a classification branch and a regression branch: the classification branch outputs the target class confidence score for each grid position, expressed as: The regression branch outputs the core parameters of the target, including the coordinates of the detection box center, the bounding box size, and the rotation angle, expressed as follows: Based on multiple pre-set anchor box parameters adapted to different target types, the parameters output by the regression branch are combined with the anchor boxes, and the original detection box set for each frame is obtained through decoding. This completes the detection and decoding process of the decoding network.

8. The target detection method based on adaptive integration of point cloud density according to claim 7, characterized in that: Based on continuous frame detection results and kinematic correction parameter models, precise optimization of the detection box position and volume is achieved through trajectory establishment, velocity estimation, and temporal smoothing correction. This includes establishing a target trajectory based on the intersection-over-union ratio (IoU), performing uniform Kalman filtering velocity estimation, and a temporal smoothing shape correction process. The process of establishing the target trajectory based on IoU is as follows: First, obtain continuous The original detection box set of the frame Set confidence threshold Filter out The low-confidence detection boxes are used to obtain the preprocessed set of detection boxes. };Calculate the first Frame detection box With the Frame detection box Intersection over Union (IOU) measures the degree of overlap between two bounding boxes. In the formula, Indicates the volume of the detection frame. This represents the intersection region of the two detection frames. The region is the union of regions; Set IOU matching threshold IOU th If the two detection boxes If the target categories are consistent, then an association is established between the two, and they are determined to be the same target. For consecutive... The above matching operation is repeated for each frame to form a continuously associated target trajectory: .

9. The target detection method based on adaptive integration of point cloud density according to claim 8, characterized in that: The Kalman filter parameters preset in the kinematic correction parameter model are called, and velocity estimation is completed through a prediction-update iterative process based on the continuous detection box data of the target trajectory. The process is as follows: Prediction phase: based on the first Posterior state estimation of frames and state transition matrix Calculate the first Prior state estimation of frames and prior error covariance matrix : In the formula, For the first The posterior error covariance matrix of the frame. This is the transpose of the state transition matrix. The process noise covariance matrix; Update phase: based on the first The center coordinates of the frame detection box are the observed values. Calculate Kalman gain : In the formula, For the observation matrix, For the transpose of the observation matrix, To observe the noise covariance matrix, Perform matrix inversion; correct the prior estimate based on Kalman gain to obtain the first... Posterior state estimation of frames and posterior error covariance matrix : In the formula, For the identity matrix, from Extracting velocity components As the first The velocity estimate of the target in the frame; Used to characterize the confidence level of the estimation results.

10. The target detection method based on adaptive integration of point cloud density according to claim 9, characterized in that: Combining velocity estimates with multi-frame temporal information, position offset correction and volume expansion suppression are performed respectively, and the processes include: For position offset correction, the theoretical position of the target is first predicted based on the velocity estimate output by the Kalman filter, and the initial position offset is obtained by subtracting the original detection box center coordinates from this estimate. , Introducing the posterior error covariance matrix Calculate the confidence weighting coefficient : In the formula, This is a matrix trace operation, where C is a preset calibration constant. The initial offset is corrected using weighted coefficients. For continuous The weighted offsets of the frames are averaged to obtain the time-smoothed offsets. , To reduce noise interference in a single frame, calculate the velocity change between adjacent frames. ,like Then, the smoothed offset is adjusted for trend adaptation, and finally the adjusted offset is superimposed on the original detection box center coordinates to obtain the corrected center coordinates. In the formula, , The coordinates of the original detection box center are: , This is the offset after trend adaptation. For volume expansion suppression, the target velocity modulus is first calculated. Speed ​​levels are determined based on speed grading thresholds from a kinematically corrected parameter model. Call the mapping function Matching yields volume correction coefficients ; in continuous The average volume of the original detection bounding box in the frame is used as the temporal volume benchmark. Calculate the initial correction volume: Calculate the initial corrected volume and the original volume of the current frame. volume change rate Verify the rationality of the correction. like Then a conservative correction factor is used. Recalculate: ;like Then, temporal smoothing is performed on the corrected volume of multiple consecutive frames to obtain the final corrected volume. ; By fusing the corrected center coordinates and volume information, an accurate final detection bounding box is output.

Images