A three-dimensional target detection method and system based on a laser radar point cloud

Abstract

The application provides a kind of three-dimensional target detection method and system based on laser radar point cloud, it is related to target detection technical field, including: obtaining single frame three-dimensional point cloud data and carrying out voxelization encoding, generate two-dimensional projection feature and key point initial feature;Based on the local space curvature of key point dynamically determines neighborhood search range, obtains near neighbor point set;Based on local data density and surface normal vector angle, the near neighbor point set is reweighted, and high-dimensional local feature is obtained;Extract macroscopic contour feature and microscopic relative position feature to generate feature gating chart, and filter high-dimensional local feature using feature gating chart, to obtain key point fusion feature;Based on two-dimensional projection feature, generate three-dimensional space candidate region, and fill key point fusion feature to corresponding three-dimensional space candidate region, to obtain the feature of three-dimensional space candidate region after filling;Based on the feature of three-dimensional space candidate region after filling, execute target identification and parameter prediction, output three-dimensional target detection result.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a three-dimensional target detection method and system based on lidar point clouds. Background Technology

[0002] A lidar point cloud is a set of discrete three-dimensional spatial data points calculated by lidar based on information such as the return time, direction, and intensity of the laser beam emitted into the surrounding environment. Each point typically contains three-dimensional coordinates, and some devices also record attributes such as reflection intensity. A large number of points together constitute the three-dimensional geometry of the target and its surrounding environment. Compared to ordinary two-dimensional images, lidar point clouds can directly provide more accurate distance, spatial location, and geometric structure information, and are therefore widely used in autonomous driving, robotics, intelligent transportation, and three-dimensional environmental perception. Three-dimensional target detection refers to the technology of identifying target categories and determining the target's position, size, and orientation in three-dimensional space from point clouds, images, or multimodal sensor data. Taking lidar point clouds as an example, three-dimensional target detection algorithms need to extract effective features from a large number of disordered, sparse, and unevenly distributed points to determine whether there are targets such as vehicles, pedestrians, and cyclists, and represent the target's position and spatial range using a three-dimensional bounding box. A three-dimensional detection box can typically be described by parameters such as the target's center coordinates, length, width, height, and rotation angle.

[0003] 3D object detection is a crucial technology for environmental perception in fields such as intelligent driving, mobile robotics, and intelligent transportation. Compared to 2D object detection, it not only identifies the objects present in the environment but also acquires information such as their position, size, distance, and orientation in real 3D space, enabling the system to more accurately understand the spatial relationships between objects and between objects and their surroundings. In autonomous driving scenarios, reliable 3D object detection provides essential data for vehicle obstacle avoidance, path planning, collision warning, and motion decision-making. Therefore, improving the accuracy, real-time performance, and robustness of 3D object detection is of great significance for enhancing the environmental perception capabilities of intelligent systems and ensuring operational safety in practical applications.

[0004] However, existing 3D target detection technologies often employ relatively fixed and uniform information extraction methods when processing point clouds. This makes it difficult to fully adapt to the differences in data distribution caused by different targets, distances, and geometric shapes. It is easy to introduce invalid or interfering information. At the same time, it does not make sufficient use of the relationship between the overall shape and detailed structure of the target. As a result, the accuracy of judging the target boundary and spatial pose is low in sparse point clouds, target occlusion, and complex scenes, which in turn affects the accuracy and stability of 3D target detection. Summary of the Invention

[0005] To address the technical problem that existing 3D target detection technologies often employ relatively fixed and uniform information extraction methods when processing point clouds, making it difficult to fully adapt to the differences in data distribution caused by different targets, distances, and geometric shapes, and easily introducing invalid or interfering information, while also failing to fully utilize the correlation between the overall shape and detailed structure of the target, resulting in low accuracy in judging the target boundary and spatial attitude in sparse point clouds, target occlusion, and complex scenes, thus affecting the accuracy and stability of 3D target detection, this invention provides a 3D target detection method and system based on lidar point clouds.

[0006] The technical solutions provided by the embodiments of the present invention are as follows: First aspect This invention provides a three-dimensional target detection method based on lidar point clouds, comprising: S1: Acquire single-frame 3D point cloud data, perform voxelization encoding, and generate 2D projection features and initial keypoint features; S2: Dynamically determine the neighborhood search range based on the local spatial curvature of key points and obtain the set of nearest neighbors; S3: Based on the local data density and the angle between the surface normal vectors, the nearest neighbor set is reweighted to obtain high-dimensional local features; S4: Extract macroscopic contour features and microscopic relative position features to generate a feature gating map, and use the feature gating map to filter high-dimensional local features to obtain key point fusion features; S5: Generate a 3D spatial candidate region based on 2D projection features, and fill the corresponding 3D spatial candidate region with key point fusion features to obtain the filled 3D spatial candidate region features. S6: Perform target recognition and parameter prediction based on the features of the filled 3D spatial candidate region, and output the 3D target detection results.

[0007] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, by voxelizing and encoding the acquired single-frame 3D point cloud data, the initial features of key points are preserved while forming 2D projection features. The neighborhood search range is dynamically determined based on the local spatial curvature of the key points, enabling the acquisition of neighborhood information to adapt to different target surface morphologies and point cloud distribution changes. Furthermore, the set of nearest-neighbor points is reweighted by combining local data density and the angle between surface normal vectors, so that information with a high degree of correlation with the geometric structure of key points is more effectively expressed, and the interference of sparse distribution, noise points, and irrelevant points on feature extraction is reduced. At the same time, a feature gating map is generated by combining macroscopic contour features and microscopic relative position features to selectively filter high-dimensional local features, so that the overall shape information of the target and the local detail information are effectively complementary. Subsequently, the 2D projection features are used to determine the 3D spatial candidate region, and the key point fusion features are filled into the corresponding region, so that the candidate target has relatively complete spatial semantic information and local geometric information, thereby providing more discriminative feature basis for target recognition and parameter prediction. It enhances the adaptability to targets of different scales, shapes, and point cloud densities, and reduces interference from invalid information in environments with sparse point clouds, target occlusion, and complex conditions, thereby improving the reliability of target boundary, spatial position, and attitude parameters, and ultimately enhancing the accuracy and stability of 3D target detection. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart illustrating a three-dimensional target detection method based on lidar point clouds provided in an embodiment of the present invention; Figure 2 A model framework diagram of a three-dimensional target detection method based on lidar point clouds provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of voxelized spatial encoding and keypoint mapping provided in an embodiment of the present invention; Figure 4 A flowchart of adaptive receptive field dynamic search calculation provided in an embodiment of the present invention; Figure 5 This is a flowchart of local feature reweighting and dimensionality enhancement provided in an embodiment of the present invention; Figure 6 This is a flowchart of a multi-branch gating fusion and purification process provided in an embodiment of the present invention. Detailed Implementation

[0010] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0011] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0012] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0013] Reference manual attached Figure 1 The diagram shows a flowchart of a three-dimensional target detection method based on lidar point clouds provided by an embodiment of the present invention.

[0014] Reference manual attached Figure 2 The diagram shows a model framework diagram of a three-dimensional target detection method based on lidar point clouds provided by an embodiment of the present invention.

[0015] This invention provides a 3D target detection method based on LiDAR point clouds. This method can be implemented by a 3D target detection device based on LiDAR point clouds, which can be a terminal or a server. The processing flow of the 3D target detection method based on LiDAR point clouds may include the following steps: S1: Acquire single-frame 3D point cloud data, perform voxelization encoding, and generate 2D projection features and initial keypoint features.

[0016] Among them, single-frame 3D point cloud data refers to a set of discrete spatial point data obtained by LiDAR scanning the surrounding environment within one acquisition cycle, used to characterize the spatial distribution of vehicles, pedestrians, roads, and other objects in the environment at the current moment. Two-dimensional projection features refer to the two-dimensional feature representation formed by extracting features from the voxelized 3D point cloud and compressing the resulting 3D voxel features along the height direction. Initial keypoint features refer to the initial feature representation formed by selecting spatially representative keypoints from the original 3D point cloud and mapping the feature information related to the spatial position of each keypoint in the two-dimensional projection features to the corresponding keypoints.

[0017] In one possible implementation, S1 specifically includes: S101: Divide a single frame of 3D point cloud data into a regular voxel grid, extract multi-scale voxel features through 3D sparse convolution, and generate 2D projection features by projecting along the height dimension.

[0018] More specifically, the single-frame 3D point cloud data acquired by the LiDAR is first regularly divided within a pre-defined 3D spatial range to form a series of uniform voxel grids. Then, a 3D sparse convolutional neural network is used to extract features from the non-empty voxels, obtaining multi-scale voxel features with different spatial granularities by stacking multiple downsampling layers. Finally, the extracted 3D voxel features are max-pooled or summed along their height dimension (usually the Z-axis) to compress all height-related feature information onto the same plane, generating a 2D bird's-eye view (BEV) projection feature, i.e., a 2D projection feature, rich in semantic information.

[0019] S102: Perform key point sampling in a single frame of 3D point cloud data to obtain a set of key point coordinates. Calculate interpolation weights based on the spatial distance between the key point coordinate set and each feature point in the 2D projection features. Map the 2D projection features to each key point through the interpolation weights to obtain the initial features of the key points.

[0020] More specifically, firstly, a sparse set of keypoint coordinates representing the overall spatial contour of the original point cloud is selected using a farthest-point sampling algorithm. Then, the Euclidean distance between each keypoint and the center of the grid of each feature point in the 2D projected feature map is calculated. The interpolation weights of each feature point are calculated based on the reciprocal of these distances and then normalized. Finally, these interpolation weights are weighted and summed with the corresponding feature point features, thereby completely and smoothly mapping the dense semantic features on the regular 2D feature map to each irregularly distributed keypoint, generating an initial feature vector with rich contextual information.

[0021] The farthest point sampling algorithm is a spatial sampling strategy that iteratively selects representative key points in a 3D point cloud. Its execution process is as follows: First, an initial point is randomly selected as the first key point. Then, in each iteration, the minimum Euclidean distance between all currently selected key points and all remaining points is calculated, and the remaining point with the largest distance value is selected as the next key point. This ensures that each newly added key point is located at the spatial position farthest from the selected point set. This process continues until a preset number of key points is reached. The final output key point set can uniformly cover the complete spatial contour of the original point cloud, effectively avoiding excessive clustering of sampling points in local high-density areas.

[0022] The formula for calculating the interpolation weights is as follows:

[0023] In the formula, For the first Interpolation weights for each feature point For key points and the first Euclidean distance between feature points The total number of feature points participating in the interpolation. To prevent the preset minimum value where the denominator is zero.

[0024] It should be noted that in this process, the single-frame 3D point cloud data is first processed by regular voxelization, transforming the originally discrete, disordered, and unevenly distributed point cloud into a spatial structure that is easy for neural networks to process. Then, 3D sparse convolution is used to extract multi-level features only from non-empty voxels, reducing unnecessary spatial computation while obtaining multi-scale voxel features under different receptive fields. Subsequently, max pooling or summation is performed on the 3D voxel features along the height dimension, aggregating spatial information from different height positions onto a unified bird's-eye view plane, thus forming a 2D projection feature that combines scene spatial structure and semantic information. Based on this, points farthest from existing keypoints are gradually selected from the original point cloud through farthest point sampling, ensuring that a limited number of keypoints can cover the entire point cloud space relatively evenly, avoiding keypoint concentration in local high-density areas. Finally, interpolation weights are calculated based on the Euclidean distance between the keypoints and the feature points involved in the interpolation, where... Feature points closer to the keypoints receive larger initial weights, while the summation term in the denominator normalizes the weights, ensuring a consistent sum of weights used in the interpolation, and minimizing the value... This avoids the division-by-zero problem when the distance is zero. Finally, the corresponding features are weighted and summed using normalized interpolation weights, so that the semantic information in the 2D projection is smoothly mapped to the irregularly distributed keypoints according to spatial proximity. This leverages the advantages of regular voxels and 2D projection in extracting semantic information over a large range, while preserving the expressive power of keypoints for their original 3D spatial location and geometric structure. The resulting initial keypoint features possess both spatial representativeness and contextual semantic information, providing a more stable and sufficient feature foundation for subsequent local geometric analysis and feature enhancement.

[0025] S2: Dynamically determine the neighborhood search range based on the local spatial curvature of key points and obtain the set of nearest neighbor points.

[0026] The local spatial curvature of a keypoint is a parameter used to characterize the flatness or geometric abrupt changes of the local point cloud surface where the keypoint is located. The neighborhood search range refers to the spatial region in three-dimensional space centered on the current keypoint, used to find other points with local spatial relationships to that keypoint. The nearest neighbor set refers to a group of points selected from the original point cloud based on the dynamically determined neighborhood search range, located around the current keypoint and used to describe the local spatial structure of that keypoint.

[0027] In one possible implementation, S2 specifically includes: S201: For any key point, extract the three-dimensional coordinates in its initial neighborhood to construct a local covariance matrix, calculate the eigenvalues ​​of the local covariance matrix, and calculate the curvature parameters that characterize the local surface geometry of the key point based on the eigenvalues.

[0028] More specifically, in step S201, firstly, taking any key point as the center, a set of three-dimensional point coordinates of its nearest neighbors is obtained according to a preset initial neighborhood range. Then, the mean vector of all point coordinates within this set is calculated, and a local covariance matrix reflecting the spatial distribution characteristics of the point group is constructed. Next, eigenvalue decomposition is performed on the covariance matrix to obtain its three eigenvalues ​​arranged in descending order. Finally, based on the numerical relationship of these three eigenvalues, the ratio of the smallest eigenvalue to the sum of the three eigenvalues ​​is used as a curvature parameter to quantitatively evaluate whether the local surface where the key point is located is a flat region or a geometrically abrupt edge.

[0029] The formula for calculating the curvature parameter is as follows:

[0030] In the formula, For curvature parameters, , , Let be the three eigenvalues ​​of the local covariance matrix, and satisfy . .

[0031] S202: Based on the comparison results between the curvature parameter and the preset curvature threshold, dynamically determine the search radius corresponding to the key point, and use the search radius to construct a spatial query operator to filter the set of nearest neighbors.

[0032] More specifically, based on the assigned search radius, the spherical query operator is invoked to draw a spherical region in three-dimensional space with the key point as the center and a dynamically determined search radius. The coordinates and feature data of all neighboring points completely located within this sphere are then filtered and output to form the final set of nearest neighbors.

[0033] It should be noted that in this process, firstly, taking each key point as the center, the 3D coordinates of surrounding points within a pre-defined initial neighborhood are obtained. Then, the spatial discretization relationship of these coordinates relative to the mean position is used to construct a local covariance matrix, enabling this matrix to reflect the distribution degree of the neighborhood point cloud along different spatial directions. Subsequently, eigenvalue decomposition is performed on the local covariance matrix, obtaining three eigenvalues. The larger eigenvalues ​​correspond to the degree of variation in the main extension direction of the point cloud, while the smallest eigenvalue reflects the degree of deviation of the point cloud from the main local surface. Therefore, the smallest eigenvalue is used... Calculating curvature parameters allows us to characterize the geometric complexity of a local surface by utilizing the proportion of the minimum directional change in the overall spatial change. When the point cloud is mainly distributed on an approximate plane, the dispersion perpendicular to that plane is relatively small, resulting in relatively small minimum eigenvalues ​​and curvature parameters. However, at edges, corners, or other locations with abrupt geometric changes, the dispersion of the point cloud increases in multiple spatial directions, leading to a corresponding increase in the curvature parameter. Therefore, using this parameter to distinguish between flat regions and regions with abrupt geometric changes provides a clear spatial geometric basis. After obtaining the curvature parameter, it is compared with a preset curvature threshold. Based on this, the search radius is dynamically configured for different key points, and a corresponding spherical query space is established with the key points as the center. The coordinates and features of the points within this space are then filtered to form a set of nearest neighbors. Compared to applying a uniform neighborhood scale to all keypoints, this approach can adjust the information collection range based on local geometry. This allows for more constrained neighborhoods to be used in locations with significant geometric changes to reduce the mixing of cross-boundary points and background points, while flatter locations can utilize more surrounding information to describe continuous surfaces. This improves the consistency between the set of nearest neighbors and the true local structure of the keypoints, providing more accurate and targeted data for subsequent feature reweighting and local geometric information extraction.

[0034] In one possible implementation, step S202, dynamically determining the search radius corresponding to the key point based on the comparison result of the curvature parameter and the preset curvature threshold, specifically includes: S2021: When the curvature parameter is greater than the preset curvature threshold, the search radius is determined as the first preset radius.

[0035] S2022: When the curvature parameter is less than or equal to the preset curvature threshold, the search radius is determined as the second preset radius, which is greater than the first preset radius.

[0036] Understandably, for each keypoint with a calculated curvature parameter, the curvature parameter is compared with a pre-set fixed threshold: if the curvature parameter is greater than the threshold, the keypoint is determined to be located in a region of geometric abrupt change, and a smaller first preset radius is dynamically assigned to it. If the curvature parameter is less than or equal to the threshold, the keypoint is determined to be located in a flat region, and a larger second preset radius is dynamically assigned to it.

[0037] This process adaptively adjusts the neighborhood search scale by comparing the curvature parameters of each key point with a preset curvature threshold: for regions with large curvature abrupt changes in geometry, a smaller first preset radius is used, concentrating the search area near the key points and reducing the possibility of introducing other objects or background points across the target boundary. For flat regions with small curvature, a larger second preset radius is used to obtain more neighboring points with similar geometric properties, supplementing local structural information. This approach matches the neighborhood scale with the geometric complexity of the local surface, and compared to a fixed search radius, it balances the feature purity of edge regions with the information sufficiency of flat regions, thereby improving the ability of the nearest neighbor set to represent the true local geometric structure.

[0038] S3: Based on the local data density and the angle between the surface normal vectors, the set of nearest points is reweighted to obtain high-dimensional local features.

[0039] Local data density is an indicator used to characterize the density of point cloud data distribution within a finite spatial range around a keypoint. Surface normal vector angle refers to the spatial angle formed between the normal vector of the local surface where the keypoint is located and the normal vector of the local surface where one of its nearest neighbors is located; it measures the degree of alignment of the local surfaces of the two points. High-dimensional local features refer to high-dimensional feature representations that comprehensively characterize the keypoint's own information as well as the local geometric, spatial distribution, and semantic information of its surrounding nearest neighbors.

[0040] In one possible implementation, S3 specifically includes: S301: Count the actual number of nearest neighbors within the neighborhood search range, and determine the density weight based on the actual number of nearest neighbors.

[0041] In one possible implementation, determining the density weight based on the actual number of nearest neighbors in step S301 specifically includes: S3011: Obtain the maximum expected number of points within the neighborhood search range, and calculate the sparsity index based on the ratio of the actual number of nearest neighbor points to the maximum expected number of points.

[0042] S3012: Generate density weights that are negatively correlated with the actual number of nearest neighbors based on sparsity index.

[0043] More specifically, for the neighborhood search range determined by the spherical query, the number of nearest neighbors within this spatial range is counted and obtained. This number is compared with the predefined maximum expected number of points to calculate its ratio. This ratio is then input into a preset nonlinear mapping function for transformation, ultimately generating a density weight coefficient that monotonically increases as the actual number of nearest neighbors decreases. This achieves numerical compensation for sparse features caused by long distances or occlusion, forcing the network to focus on weak signals in such areas.

[0044] It should be noted that a sparsity index is obtained by statistically analyzing the actual number of nearest neighbors within the current neighborhood search range and comparing it with the maximum expected number of points. This index quantifies the density of the point cloud data surrounding the current key point. Subsequently, a pre-defined nonlinear mapping relationship is used to convert the sparsity index into density weights, such that a smaller number of actual nearest neighbors corresponds to a larger density weight, while a larger number of nearest neighbors corresponds to a smaller density weight. This method leverages the characteristic that the number of neighborhood points directly reflects the sufficiency of local point cloud sampling, adaptively compensating for sparse regions caused by factors such as distance and occlusion, while avoiding excessive enhancement of feature responses in dense regions. This reduces the impact of uneven point cloud density on local feature extraction and improves the balance and stability of feature representation in regions of different densities.

[0045] S302: Calculate the angle between the key point and the surface normal vector of each nearest neighbor in the set of nearest neighbors, and determine the attenuation penalty coefficient based on the angle.

[0046] More specifically, for each selected nearest neighbor point, the local surface normal vector of that point and the local surface normal vector of the central key point are first calculated. Then, the spatial angle difference between these two normal vectors is calculated, and a pre-defined monotonically decreasing nonlinear function is applied to this angle difference for numerical transformation. Finally, the transformed output value is determined as the attenuation penalty coefficient for that nearest neighbor point. This ensures that the larger the angle between the normal vectors (the greater the difference in surface orientation), the smaller the attenuation penalty coefficient is obtained for the nearest neighbor point. This suppresses feature response in the network, reducing the interference contribution of background clutter from different physical surfaces to the final feature fusion.

[0047] The formula for calculating the attenuation penalty coefficient is as follows:

[0048] In the formula, For the first The attenuation penalty coefficient of the nearest neighbor points, For the set of key points and nearest neighbors, the first... The angle between the surface normal vectors of the nearest neighbor points These are the preset attenuation control parameters.

[0049] S303: The nearest neighbor features are weighted and fused using density weights and decay penalty coefficients, and then the high-dimensional local features are obtained by increasing the dimensionality through neural network layers.

[0050] More specifically, the calculated density weights and attenuation penalty coefficients are multiplied element-wise to generate a comprehensive modulation factor for each nearest neighbor. This modulation factor is then multiplied element-wise with the original feature tensors corresponding to the nearest neighbors to achieve numerical amplification and suppression adjustment of the original features. Subsequently, the modulated features of all nearest neighbors are concatenated with the features of the central keypoint itself to form an enhanced local feature vector. Finally, this enhanced vector is input into a pre-defined multilayer perceptron, where it undergoes high-dimensional nonlinear mapping and feature encoding through fully connected layers, nonlinear activation functions, and batch normalization layers, outputting a high-dimensional local feature tensor representing local spatial geometry and semantic information.

[0051] It should be noted that this process first counts the actual number of nearest neighbors within the search range corresponding to each keypoint, and determines the density weight based on the local data density reflected by the number of points, so that effective features in sparser areas of the point cloud can be appropriately compensated. Simultaneously, the normal vectors of the local surfaces where the central keypoint and each nearest neighbor point are located are estimated, and the angle between them is calculated to characterize the consistency of the nearest neighbor points and keypoints in surface orientation. The attenuation penalty coefficient... In the middle, when the angle between the normal vectors When smaller, Approaching one makes the exponential term approach zero, resulting in a decay penalty coefficient approaching one, thus preserving more sufficiently the features of nearest neighbors with similar surface orientations to the keypoint. As the included angle increases, Decreasing the exponential term increases the absolute value of the exponential term, causing the attenuation penalty coefficient to gradually decrease, thereby weakening the characteristic response that may originate from other object surfaces, boundary regions, or background clutter. The parameter... (A value of 1 or 3 can be used to adjust the sensitivity of this attenuation effect to changes in the included angle. Based on this, the density weight and attenuation penalty coefficient are combined to form a comprehensive modulation factor, which differentiates the features of each nearest neighbor point, then concatenates them with the features of the central key point itself, and completes nonlinear mapping and up-dimensional encoding through a multilayer perceptron to obtain high-dimensional local features. The reliability of this point cloud feature processing is not only affected by the sampling density, but also closely related to whether the neighboring points belong to the same or continuous physical surface. By simultaneously adjusting the contribution of nearest neighbors from both the data distribution and surface geometric consistency dimensions, it is possible to enhance the effective information in sparse regions while suppressing interference information with weak geometric correlation. This results in high-dimensional local features that more centrally express the local spatial structure and semantic information of the real target, improving the accuracy and stability of subsequent feature fusion and 3D target detection under sparse, occluded, and complex background conditions.

[0052] S4: Extract macroscopic contour features and microscopic relative position features to generate a feature gating map, and use the feature gating map to filter high-dimensional local features to obtain key point fusion features.

[0053] Among them, macroscopic contour features refer to feature information extracted from high-dimensional local features to characterize the overall geometric shape, spatial topological relationships, and target category attributes of the keypoint's neighborhood. Microscopic relative position features refer to feature representations used to describe the fine-grained spatial positional relationships and feature differences between the keypoint and its nearest neighbors. The feature gating map is a weight distribution generated jointly from macroscopic contour features and microscopic relative position features, used to control the degree of information retention in high-dimensional local features. Keypoint fusion features refer to the keypoint feature representations obtained by selectively modulating high-dimensional local features using the feature gating map.

[0054] In one possible implementation, S4 specifically includes: S401: Extracts macroscopic contour features of high-dimensional local features through continuous convolution operations.

[0055] More specifically, the generated high-dimensional local feature tensor is input into a feature extractor consisting of a preset number of two-dimensional convolutional layers stacked sequentially. After each convolution, it is transformed by a non-linear activation function and a batch normalization layer, thereby gradually expanding the receptive field while aggregating and refining the global morphology and spatial topology information of the point cloud in the neighborhood of key points. Finally, a macroscopic contour feature tensor representing the overall contour and category attributes of the target is output.

[0056] S402: Calculate the 3D coordinate difference and feature difference between each nearest point in the nearest point set and the key point, and obtain the micro-relative position features through multilayer perceptron encoding.

[0057] More specifically, for each nearest neighbor selected within the neighborhood of the keypoint, the relative position vector between it and the central keypoint in three-dimensional space is first calculated, and the difference between their feature vectors in the high-dimensional local feature space is also calculated. Then, the coordinate difference vector and the feature difference vector are concatenated along the feature dimension to form an input tensor that comprehensively describes the geometric and semantic relationships between point pairs. Finally, this input tensor is fed into a pre-defined multilayer perceptron, which performs high-dimensional nonlinear transformation and encoding through its internal fully connected layers and nonlinear activation functions, outputting microscopic relative position features that can finely characterize details such as local surface convexity and edge orientation.

[0058] In one possible implementation, S402 specifically includes: S4021: Calculate the coordinate difference vector between the 3D spatial coordinates of each nearest neighbor in the nearest neighbor set and the 3D spatial coordinates of the key point, and the feature difference vector between the features of each nearest neighbor in the nearest neighbor set and the features of the key point.

[0059] S4022: Concatenate the coordinate difference vector and the feature difference vector, input them into a multilayer perceptron for nonlinear transformation, and obtain the micro-relative position sub-features corresponding to each nearest neighbor point.

[0060] S4023: Perform pooling operation on the micro-relative position sub-features of all nearest neighbor points to obtain the micro-relative position features.

[0061] The formula for calculating the microscopic relative position sub-feature is as follows:

[0062] In the formula, For the first Microscopic relative position sub-features of nearest neighbor points This represents the multilayer perceptron operation. For the first The coordinate difference vector between the nearest neighbor points and the key point For the first The feature difference vector between the nearest neighbor points and the key point This indicates vector concatenation.

[0063] It should be noted that this process first calculates the three-dimensional coordinate difference vector between the nth nearest neighbor point and the central key point. and feature difference vector The coordinate difference describes the distance, direction, and spatial offset of neighboring points relative to the keypoint, while the feature difference reflects the differences in their local geometric and semantic representations. The two types of differences are then concatenated and processed according to... The input to a multilayer perceptron undergoes nonlinear mapping, enabling the network to learn the correlation between spatial location changes and feature changes, thereby obtaining microscopic relative position sub-features corresponding to each nearest neighbor point. Finally, all sub-features are pooled and aggregated to form a unified microscopic relative position feature. This approach utilizes both geometric position differences and feature representation differences. Compared to using only absolute coordinates or single feature information, it can more fully characterize the relative spatial structure, surface undulations, and edge orientation within the neighborhood of keypoints. Furthermore, pooling reduces the impact of differences in the number and arrangement of nearest neighbors, thereby improving the completeness, stability, and discriminative ability of local detail feature representation.

[0064] S403: Combine macroscopic contour features with microscopic relative position features, generate a feature gating map through a gating activation function, and multiply the feature gating map element-wise with high-dimensional local features to obtain key point fusion features.

[0065] The formula for calculating the feature gating map is as follows:

[0066] In the formula, For feature-gated graphs, It is the Sigmoid activation function. and These are the weight matrix and bias term of the gate layer, respectively. For macroscopic outline features, For microscopic relative positional features, This indicates a feature splicing operation.

[0067] More specifically, in step S403, the macroscopic contour feature tensor extracted in step S401 and the microscopic relative position feature tensor output in step S402 are concatenated and fused along the feature dimension. Then, the concatenated fused feature vector is input to a preset gated activation function layer. After internal nonlinear transformation operations, a weight distribution matrix with a value range of 0 to 1, i.e., a feature gating map, is generated. Finally, this feature gating map is multiplied element-wise and spatially by the high-dimensional local feature tensor generated in step S3. This enhances and preserves the features of the true target's geometric shape region while numerically suppressing background clutter features that are prone to false detection. The final output is a filtered and purified keypoint fused feature tensor.

[0068] It should be noted that this process first performs continuous convolution on the obtained high-dimensional local features, expanding the receptive field layer by layer and aggregating feature information within the neighborhood to extract macroscopic contour features that reflect the overall geometric shape, contour structure, and spatial topological relationships of the target. Simultaneously, centered on the keypoint, the 3D coordinate difference and high-dimensional feature difference between each nearest neighbor point and the keypoint are calculated, and these are concatenated and input into a multilayer perceptron for nonlinear encoding. This allows the obtained microscopic relative position features to reflect fine-grained information such as the spatial offset of nearest neighbors relative to the keypoint, local surface undulations, and edge orientation. Subsequently, the macroscopic contour features are... Microscopic relative positional features To splice together, and according to Generate a feature-gated map, where the weight matrix and bias terms The network can be trained adaptively to learn the combination relationship between macroscopic and microscopic information. The sigmoid activation function restricts the gate value to between zero and one, allowing different channels and spatial locations to obtain different degrees of feature preservation weights. Finally, the feature gating map is multiplied element-wise with the original high-dimensional local features, so that target-related features with larger gate values ​​are more fully preserved, while background clutter, local anomalies, and features with low correlation to the target structure are suppressed with smaller gate values. It is understandable that relying solely on the overall contour can easily ignore local details such as target edges, while relying solely on local relative relationships is easily interfered with by noise and local similar structures. By using macroscopic contour information and microscopic relative position information to jointly determine the feature gating weight, high-dimensional local features can be selectively filtered from two levels: overall structural consistency and local geometric correlation. This ensures that the final keypoint fusion feature retains the overall shape information of the target while highlighting the local geometric details that have a discriminative effect, thereby reducing the interference of irrelevant background features on the construction of subsequent candidate region features and improving the ability of 3D target detection to represent target boundaries, spatial morphology, and category attributes.

[0069] S5: Generate a 3D spatial candidate region based on 2D projection features, and fill the corresponding 3D spatial candidate region with key point fusion features to obtain the filled 3D spatial candidate region features.

[0070] Among them, the three-dimensional spatial candidate region refers to the spatial range in three-dimensional physical space that may contain the target to be detected, which is initially determined based on two-dimensional projection features. The filled three-dimensional spatial candidate region feature refers to the feature representation used to uniformly describe the candidate target region after filtering and aggregating the key point fusion features belonging to the corresponding three-dimensional spatial candidate region.

[0071] In one possible implementation, S5 specifically includes: S501: Input the 2D projection features into the region proposal network to generate multiple initial 3D candidate boxes.

[0072] More specifically, the process inputs the generated 2D projection (bird's-eye view, BEV) features with global semantic information into a pre-trained Region Proposal Network (RPN). This network, through multi-layer convolutional and fully connected layers, simultaneously outputs the confidence score of each grid cell containing the target and the parameter offset of the initial 3D candidate boxes. Then, it filters out results with confidence scores higher than a preset threshold, decodes and generates multiple sets of initial 3D candidate boxes of different sizes and spatial locations, providing candidate regions for subsequent region of interest feature extraction.

[0073] S502: Determine the 3D spatial candidate region corresponding to each initial 3D candidate box, and use the region of interest pooling operator to aggregate the key point fusion features into the 3D spatial candidate region inside the initial 3D candidate box to obtain the filled 3D spatial candidate region features.

[0074] More specifically, firstly, based on the regression parameters of each initial 3D candidate box generated by the region proposal network, its corresponding cuboid spatial boundary in 3D physical space is decoded, thus determining the 3D spatial candidate region corresponding to the candidate box. Then, the region of interest pooling operator is invoked to iterate through all keypoint fusion features refined in the aforementioned steps. An indicator function is used to determine whether each keypoint is within the spatial boundary of the current candidate box. Average pooling aggregation is performed on all keypoint fusion features falling within the boundary range, summarizing the scattered point-level features into a fixed-dimensional feature tensor that uniformly represents the 3D spatial target candidate region. Finally, a fully filled 3D spatial candidate region feature with both global semantics and local detail information is obtained.

[0075] The feature aggregation formula for the region-of-interest pooling operator is as follows:

[0076] In the formula, The features of the candidate regions in the filled 3D space. This is the set of key points within the initial 3D candidate bounding box. For the number of key points, For set The Middle Key point fusion features of each element The three-dimensional spatial boundary of the initial three-dimensional candidate box. This is an indicator function.

[0077] It should be noted that in this implementation, the obtained two-dimensional projection features are first input into the region proposal network. Utilizing the target semantic information and horizontal spatial distribution information contained in the two-dimensional bird's-eye view features, the network predicts the probability of a target existing at each grid location and the position, size, and other parameters of the corresponding 3D candidate boxes. Multiple initial 3D candidate boxes are obtained through screening based on a confidence threshold and parameter decoding, thereby narrowing the subsequent processing scope from the entire point cloud scene to local 3D spatial regions where targets may exist. Subsequently, the corresponding cuboid spatial boundary is determined based on the parameters of each initial 3D candidate box. The region-of-interest (ROI) pooling operator is used to determine whether keypoints fall within the boundary. For keypoints located within the candidate region, their keypoint fusion features, obtained through the aforementioned geometric enhancement and gating filtering, are aggregated. (Formula) Indicator functions This serves as a spatial filter, allowing keypoints to participate in feature aggregation when their features are within the candidate box boundaries, otherwise they are excluded from feature construction for the current candidate region. The number of keypoints is then utilized. The effective features are averaged so that even if different candidate regions contain different numbers of key points, they can form a region-level feature representation with a relatively stable scale and fixed dimensions.

[0078] Understandably, 2D projection features have a large receptive field, making them suitable for quickly determining the location of potential targets. Keypoint fusion features, on the other hand, retain more detailed 3D geometric structures and local semantic information. Establishing a spatial correspondence between the two through candidate regions allows for information bridging from global localization to fine-grained local representation. Furthermore, aggregating only keypoint features within the candidate bounding box reduces interference from background points outside the candidate region, while average pooling reduces the impact of differences in the number of local point clouds on feature amplitudes. Therefore, the resulting filled 3D spatial candidate region features can balance the overall semantic information of the target with the details of the local 3D structure, providing a more concentrated, stable, and discriminative regional feature foundation for subsequent target category recognition and accurate prediction of 3D bounding box parameters.

[0079] S6: Perform target recognition and parameter prediction based on the features of the filled 3D spatial candidate region, and output the 3D target detection results.

[0080] Among them, the three-dimensional target detection result refers to the final detection information used to characterize the target category and its three-dimensional spatial state in the scene, which is obtained by classifying and predicting the spatial parameters of the candidate target based on the features of the filled three-dimensional spatial candidate region and after filtering the redundant results.

[0081] In one possible implementation, S6 specifically includes: S601: Input the filled 3D spatial candidate region features into the detection head, and output the target category probability and 3D bounding box regression parameters.

[0082] More specifically, the candidate region features obtained in step S502, after being filled and representing complete spatial information, are input into a pre-configured fully connected neural network, i.e., a detection head. This detection head contains two parallel sub-network branches. The first branch outputs the probability of the target belonging to each category by calculating the category score tensor. The second branch outputs the bounding box parameter tensor representing the precise geometric position of the target through regression calculation. The parameters include at least the three-dimensional coordinate offset of the target's center point, the length, width, and height of the three-dimensional bounding box, and the yaw angle around the vertical axis.

[0083] S602: Calculate the target center point offset, target size parameters, and target yaw angle based on the 3D bounding box regression parameters to generate a predicted 3D bounding box set.

[0084] More specifically, for each filled 3D candidate region feature, the 3D bounding box regression parameter tensor output by the detection head is parsed into three distinct sets of physical quantities: the first set is the coordinate offset of the center point on the x, y, and z axes, used to correct the center position of the candidate box. The second set is the actual length, width, and height values ​​of the target. The third set is the target's horizontal yaw angle. Subsequently, based on the preset anchor center coordinates and basic dimensions of the initial 3D candidate box, geometric decoding is performed using the above regression parameters through preset addition and multiplication operations, ultimately generating a set of predicted 3D bounding boxes that characterize the precise 3D spatial position and orientation of the predicted target.

[0085] S603: Calculate the intersection-union ratio (IUGR) between predicted 3D bounding boxes in the predicted 3D bounding box set, remove redundant boxes through non-maximum suppression, and output the 3D target detection results.

[0086] More specifically, this process calculates the ratio of the overlap volume to the union volume of any two predicted boxes in the 3D physical space based on the precise geometric parameters of each box in the predicted 3D bounding box set generated in step S602, thus obtaining the cross-union matrix. Subsequently, all predicted boxes are sorted in descending order according to a preset confidence score, and the box with the highest score is selected as the retained box. A pre-set cross-union threshold is then used for comparison, and all other predicted boxes with a cross-union ratio higher than the threshold are identified as redundant boxes and removed. This process is iterated until all predicted boxes have been filtered, and finally, the unique 3D bounding box with the highest confidence score and no overlap is output as the final 3D target detection result for each independent target in the current scene.

[0087] The formula for calculating the intersection-union ratio is:

[0088] In the formula, To predict 3D bounding boxes With the prediction of 3D bounding boxes The crossover ratio between them Let the volume be the intersection of two predicted 3D bounding boxes A and B in 3D space. Let A be the union volume of two predicted 3D bounding boxes A and B in 3D space.

[0089] It should be noted that in this process, the obtained filled 3D spatial candidate region features are first input into the detection head. Two parallel sub-networks are used to complete target category recognition and 3D bounding box parameter regression, respectively. The category recognition branch outputs the probability that the candidate region belongs to different target categories, while the bounding box regression branch predicts parameters such as the offset of the target center point in the 3D coordinate axis direction, target length, width, height, and yaw angle. Subsequently, using the center position and basic dimensions of the initial 3D candidate boxes as geometric references, the position, size, and orientation of the candidate boxes are corrected and decoded according to the above regression parameters. This makes the candidate boxes, which were originally only used for rough target localization, more closely approximate the true 3D spatial boundary of the target, thereby generating a set of predicted 3D bounding boxes with category confidence and accurate spatial geometric parameters. Since the same real target may correspond to multiple predicted boxes with similar positions and sizes, the intersection-over-union ratio (IoU) is further used to determine the degree of redundancy. The spatial overlap between two predicted 3D bounding boxes is measured by the ratio of their intersection volume to their union volume; a higher IoU indicates that the two predicted boxes are more likely to correspond to the same target. Based on this, predicted boxes are retained in descending order of confidence, while other predicted boxes with an IoU higher than a preset threshold with the current high-confidence predicted box are suppressed as redundant results until all predicted boxes have been filtered.

[0090] The candidate region features have integrated the overall semantic information of the target and local 3D geometric details, providing sufficient feature basis for both category judgment and spatial parameter regression. Bounding box regression further refines the target's position, size, and orientation based on the initial candidate boxes, while non-maximum suppression effectively solves the problem of multiple duplicate detection boxes for the same target. Therefore, this process can ultimately transform the extracted and fused point cloud features into detection results with clear target category, 3D center position, spatial size, and orientation information. This reduces duplicate detections and false detections while improving the accuracy of 3D target localization and the reliability and stability of the final detection results.

[0091] The following diagram illustrates the specific process of this solution in practical application: Refer to the instruction manual appendix Figure 3 The diagram illustrates a voxelized spatial encoding and keypoint mapping method provided by an embodiment of the present invention.

[0092] Figure 3 In the diagram, Branch A is the Voxelization Branch. This branch is mainly responsible for transforming the original, discrete 3D point cloud into 2D bird's-eye view features with richer scene semantic information. The diagram shows six processing steps / results, labeled 1 to 6.

[0093] Label 1: Spatial Cropping. This step first crops the original 3D point cloud of a single frame according to a pre-defined 3D detection spatial range, removing points outside the detection range and retaining only the point cloud data in the valid spatial region. Its purpose is to limit the spatial range required for subsequent voxelization and feature extraction.

[0094] Label 2: Regular Voxel Partition. This process regularly partitions the 3D space containing the spatially clipped point cloud according to a preset voxel size, forming a uniform 3D voxel mesh. Originally, the point cloud is discrete, disordered, and unevenly distributed. This process establishes a regular spatial structure, providing input for subsequent 3D sparse convolution. The manual describes this as "performing regular partitioning within a pre-defined 3D spatial range to form a series of uniform voxel meshes."

[0095] Label 3: 3D Sparse Convolutional Network (3D Sparse CNN). After regular voxelization, non-empty voxels are input into the 3D Sparse CNN for feature extraction. Sparse convolution is used here, meaning that computation is primarily performed on non-empty voxels with valid point cloud data, rather than performing equal computation on all voxels in the entire 3D space. The figure also labels "Layer-by-Layer Feature Downsampling" after "3D Sparse Convolutions," indicating that the 3D sparse convolutional network does not perform a single feature transformation, but rather obtains voxel features at different spatial granularities and receptive fields through layer-by-layer convolution and downsampling. The specification also explicitly describes this as "obtaining multi-scale voxel features containing different spatial granularities by stacking multiple downsampling layers."

[0096] Label 4: Multi-scale Voxel Feature Tensors. The figure uses multiple 3D feature blocks of different sizes and colors to represent the multi-scale voxel feature tensors output by the 3D Sparse CNN. Multi-scale can be understood in conjunction with the diagram as follows: different network layers have different spatial granularities and receptive fields, thus the obtained features describe the point cloud structure at different scales. Ultimately, it does not rely solely on a single scale, but forms a three-dimensional voxel feature representation containing information from different scales.

[0097] Mark 5: Height Projection Compression. After obtaining the 3D voxel features, aggregate processing such as max pooling or summation is performed on the 3D features along the height direction, commonly known as the Z-axis, to compress information from different height positions onto a unified horizontal 2D plane.

[0098] Label 6: Generate a 2D Bird's Eye View (BEV) Feature Matrix. This is the main output of branch A. After height projection compression, the original 3D voxel features form a 2D bird's eye view feature matrix, i.e., the BEV Feature Matrix. As can be seen from the heatmap in the figure, different locations have different feature responses. The BEV features preserve the positional distribution of the target in horizontal space while also containing the contextual semantic information extracted by the 3DSparse CNN. Therefore, it can be used for keypoint feature assignment and candidate target localization in the entire detection network.

[0099] Branch B: Keypoint Sampling Branch. Synchronized with Branch A, the original point cloud also enters Branch B: Keypoint Sampling. The purpose of this branch is not to rebuild regular voxels, but to directly select representative 3D keypoints from the original 3D point cloud and map the BEV semantic information obtained from Branch A onto these keypoints.

[0100] Label 1: Furthest Point Sampling (FPS). FPS operates directly on the original point cloud. Its basic process is: first, an initial point is selected; then, the distance between the remaining points and the currently selected set of keypoints is continuously calculated, prioritizing points farther from existing keypoints as new keypoints, until a predetermined number of keypoints are obtained. Therefore, the main function of FPS is to ensure good coverage of the selected keypoints across the entire 3D point cloud space, rather than concentrating them in dense local point cloud areas. Label 2: Extracting Discrete 3D Coords as Keypoints (Keypoints (Discrete 3D Coords)). After FPS, a set of discrete keypoints is obtained. The green dots and the numbers "05, 02, 07, 11, 46, 23, 17, 18, 04, 06, 23, 10" next to them are illustrative numbers / identifiers for the selected keypoints, used to represent different keypoints selected from the original point cloud; the core information is that each keypoint corresponds to a 3D spatial coordinate. Key connection between the two branches: Label 3 Feature interpolation operation. Feature Interpolation simultaneously receives BEV features generated by branch A and 3D keypoint locations generated by branch B. It calculates interpolation weights based on the Euclidean distance between the spatial location of the keypoints and the corresponding feature points in the 2D projected features, and then maps the corresponding BEV features to the keypoints according to the weights. The small curve plot next to the feature interpolation heatmap in the figure is used to help illustrate this relationship of feature interpolation / weighting as spatial distance changes.

[0101] Final output: Initial feature tensors of keypoints. After spatial Euclidean distance interpolation, the rightmost part of the graph yields the initial feature tensors of keypoints. Next to the green keypoints in the graph, further illustrative markers such as feature[v,...], fec[w,...], [x,...], and vz appear. After feature interpolation, different keypoints no longer only have three-dimensional spatial coordinates but are also assigned corresponding feature vectors / eigenvalues, thus forming the initial feature tensors of keypoints. In the graph, feature[...], fec[...], etc., are illustrative representations of the feature vectors corresponding to different keypoints; [x,...], vz, etc., are used to indicate the corresponding coordinates / feature data of the keypoints.

[0102] like Figure 3 As shown, firstly, spatial coordinate clipping and regularized voxel segmentation are performed on the input point cloud. The processed voxel data is then input into a 3D sparse convolutional neural network (3DSparseCNN). The network performs layer-by-layer 3D sparse convolution and feature downsampling calculations, outputting multi-scale voxel feature tensors. These tensors are then projected and compressed along the height dimension to generate a 2D bird's-eye view feature matrix.

[0103] Simultaneously, a farthest-point sampling algorithm is executed in the 3D point cloud coordinate space to extract a set number of discrete 3D coordinate points as keypoints. Subsequently, a feature interpolation operation is performed to map the 2DBEV features to the corresponding keypoints according to the spatial Euclidean distance, generating the initial feature tensor of the keypoints.

[0104] Refer to the instruction manual appendix Figure 4 The diagram illustrates an adaptive receptive field dynamic search calculation flowchart provided by an embodiment of the present invention.

[0105] like Figure 4 As shown, for each keypoint, a set of 3D point cloud coordinates within its initial neighborhood is extracted. By constructing a local covariance matrix and extracting eigenvalues, a curvature parameter representing the local surface geometry of the keypoint is calculated and output. Subsequently, the search radius parameter of the BallQuery operator is dynamically adjusted based on the curvature parameter. The specific mapping rule is configured as follows: when the curvature parameter is greater than a set threshold, a smaller search radius is input to the BallQuery operator to extract local edge detail features. When the curvature parameter is less than the set threshold, a larger search radius is input to the BallQuery operator to expand the receptive field and extract a wider range of point cloud features. Finally, the BallQuery operator selects the nearest neighbor points within the adjusted radius and directly outputs the coordinates and feature tensors of these points.

[0106] Refer to the instruction manual appendix Figure 5The diagram illustrates a local feature reweighting and dimensionality enhancement flowchart provided by an embodiment of the present invention.

[0107] like Figure 5 As shown, the core purpose of this step is to address the shortcomings of lidar in the real physical world, namely weak signals from distant targets and susceptibility to environmental clutter, through numerical adjustments. First, to address the issue of sparse point clouds caused by distant or obstructed targets, the network counts the actual number of points within the selected area. In real-world scanning, fewer points indicate a weaker physical signal reflected from the target. Therefore, the network employs a reverse compensation logic: fewer points result in a larger weighting coefficient, effectively "amplifying and brightening" the weak signal, forcing the network to focus on these distant sparse features (such as distant pedestrians). Conversely, more points (indicating closer distance and stronger signal) result in a smaller multiplier. To address background clutter interference, the network calculates the surface normal vectors (i.e., the physical orientation of the object's surface) of the central key point and surrounding points. In the real world, the surface orientations of objects belonging to the same group (such as the same car door or the same pedestrian) should be roughly continuous and consistent. If the calculated angle between the normal vector of a nearby point and the center point is very large, it indicates that this neighboring point is very likely a leaf, the edge of a road sign, or a protruding object on the ground that is blocking the view. For this kind of background clutter that is not on the same physical surface, the network directly assigns it an attenuation penalty coefficient close to 0, forcibly suppressing its feature response. Then, the calculated density weights and attenuation coefficient are directly multiplied with the original feature tensor to adjust the feature values. Finally, the adjusted features and coordinate data are concatenated and fed into a multilayer perceptron (MLP) for dimensionality upscaling, outputting a high-dimensional local feature tensor.

[0108] Refer to the instruction manual appendix Figure 6 The diagram illustrates a multi-branch gating fusion and purification process provided by an embodiment of the present invention.

[0109] like Figure 6 The module shown aims to effectively filter out highly confusing background clutter (such as difficult-to-bear samples like bushes or pillars resembling pedestrians) by combining macroscopic geometric features with microscopic texture features. The module employs a two-branch parallel structure: the first branch extracts macroscopic contour features through continuous convolution operations to determine the overall category attribute of the target. The second branch calculates the 3D coordinate differences and feature differences between surrounding nearest points and the central keypoint, feeding these into a multilayer perceptron to extract relative position features and using them to determine whether the surface texture and edge abrupt details of the object match the real physical target. Subsequently, the two feature paths are concatenated and input into a gating layer with a sigmoid activation function to generate a spatial attention weight map. The network then achieves adaptive feature weighting: assigning high weights to real target regions and significantly suppressing background interference that easily causes false alarms, thereby completing feature purification.

[0110] Finally, the high-dimensional keypoint fusion features of the final output are combined with the generated 2D bird's-eye view (BEV) features and input into the prediction execution unit. First, a rough outline of suspected areas containing targets is selected at a macroscopic level, and then the refined features are used for further refinement and output. First, the network feeds the 2D bird's-eye view features into the Region Proposal Network (RPN). This network quickly generates a batch of initial "3D target candidate boxes" across the entire radar scan scene. This step is equivalent to roughly outlining areas where vehicles or people might exist from a macroscopic perspective. Next, the Region of Interest (ROI) pooling operator is invoked to precisely "fill in" the refined, clutter-free keypoint features according to their actual spatial physical locations into the corresponding initial candidate boxes, assembling a complete global feature tensor for each candidate box. In this way, the originally rough candidate boxes now possess rich local detail information.

[0111] Finally, these rich candidate bounding box features are fed into a fully connected network (i.e., the detection head). The network calculates and outputs the specific classification of the target within the current bounding box (e.g., vehicle or pedestrian), as well as the target's precise geometric parameters in 3D physical space (including the 3D coordinate offset of the target's center point, and the length, width, height, and yaw angle of the 3D bounding box). In actual detection, the network often predicts multiple overlapping bounding boxes with slight deviations for the same real car or pedestrian. Therefore, the system finally calls the non-maximum suppression (NMS) operator to compare the overlap of these boxes, discarding redundant boxes and retaining only the bounding box with the highest confidence score and the most accurate result, thus outputting a clean and high-precision 3D target detection result. This avoids the limitations of static sampling and significantly reduces the false detection rate of sparse targets. It achieves targeted feature enhancement and interference suppression, improving the model's stability in harsh scenes. It avoids the information loss of traditional pooling and greatly reduces the false detection rate caused by complex background clutter.

[0112] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, the acquired single-frame 3D point cloud data is voxelized and encoded to retain the initial features of key points while forming 2D projection features. The neighborhood search range is dynamically determined based on the local spatial curvature of the key points, enabling the acquisition of neighborhood information to adapt to different target surface morphologies and point cloud distribution variations. Furthermore, the set of nearest-neighbor points is reweighted by combining local data density and the angle between surface normal vectors, allowing information highly correlated with the geometric structure of the key points to be expressed more effectively, and reducing the interference of sparse distribution, noise points, and irrelevant points on feature extraction. Simultaneously, a feature gating map is generated using macroscopic contour features and microscopic relative position features to selectively filter high-dimensional local features, effectively complementing the overall target morphology information and local detail information. Subsequently, 2D projection features are used to determine 3D spatial candidate regions, and key point fusion features are filled into the corresponding regions, ensuring that candidate targets possess both relatively complete spatial semantic information and local geometric information, thereby providing more discriminative feature basis for target recognition and parameter prediction. It enhances the adaptability to targets of different scales, shapes, and point cloud densities, and reduces interference from invalid information in environments with sparse point clouds, target occlusion, and complex conditions, thereby improving the reliability of target boundary, spatial position, and attitude parameters, and ultimately enhancing the accuracy and stability of 3D target detection.

[0113] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0114] The following points need to be explained: (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0115] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0116] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0117] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional target detection method based on lidar point clouds, characterized in that, include: S1: Acquire single-frame 3D point cloud data, perform voxelization encoding, and generate 2D projection features and initial key point features; S2: Dynamically determine the neighborhood search range based on the local spatial curvature of key points and obtain the set of nearest neighbors; S3: Based on the local data density and the angle between the surface normal vectors, the set of nearest neighbors is reweighted to obtain high-dimensional local features; S4: Extract macroscopic contour features and microscopic relative position features to generate a feature gating map, and use the feature gating map to filter the high-dimensional local features to obtain key point fusion features; S5: Generate a three-dimensional spatial candidate region based on the two-dimensional projection features, and fill the corresponding three-dimensional spatial candidate region with the key point fusion features to obtain the filled three-dimensional spatial candidate region features. S6: Perform target recognition and parameter prediction based on the features of the filled 3D spatial candidate region, and output the 3D target detection results.

2. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S1 specifically includes: S101: Divide the single-frame 3D point cloud data into a regular voxel grid, extract multi-scale voxel features through 3D sparse convolution, and project along the height dimension to generate the 2D projection features. S102: Perform key point sampling in the single frame of 3D point cloud data to obtain a set of key point coordinates. Calculate interpolation weights based on the spatial distance between the set of key point coordinates and each feature point in the 2D projection feature. Map the 2D projection feature to each key point using the interpolation weights to obtain the initial features of the key points.

3. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S2 specifically includes: S201: For any key point, extract the three-dimensional coordinates in its initial neighborhood to construct a local covariance matrix, calculate the eigenvalues ​​of the local covariance matrix, and calculate the curvature parameter characterizing the local surface geometry of the key point based on the eigenvalues. S202: Based on the comparison result between the curvature parameter and the preset curvature threshold, dynamically determine the search radius corresponding to the key point, and construct a spatial query operator with the search radius to filter the set of nearest neighbors.

4. The three-dimensional target detection method based on lidar point clouds according to claim 3, characterized in that, In step S202, dynamically determining the search radius corresponding to the key point based on the comparison result between the curvature parameter and the preset curvature threshold specifically includes: S2021: When the curvature parameter is greater than the preset curvature threshold, the search radius is determined as the first preset radius; S2022: When the curvature parameter is less than or equal to the preset curvature threshold, the search radius is determined as a second preset radius, and the second preset radius is greater than the first preset radius.

5. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S3 specifically includes: S301: Count the actual number of nearest neighbors within the neighborhood search range, and determine the density weight based on the actual number of nearest neighbors; S302: Calculate the angle between the key point and the surface normal vector of each nearest neighbor in the set of nearest neighbors, and determine the attenuation penalty coefficient based on the angle; S303: The nearest neighbor features are weighted and fused using the density weights and the attenuation penalty coefficients, and the high-dimensional local features are obtained by increasing the dimensionality through a neural network layer.

6. The three-dimensional target detection method based on lidar point clouds according to claim 5, characterized in that, In step S301, determining the density weight based on the actual number of nearest neighbors specifically includes: S3011: Obtain the maximum expected number of points within the neighborhood search range, and calculate the sparsity index based on the ratio of the actual number of nearest neighbor points to the maximum expected number of points; S3012: Generate the density weights that are negatively correlated with the actual number of nearest neighbors based on the sparsity index.

7. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S4 specifically includes: S401: Extract the macroscopic contour features of the high-dimensional local features through continuous convolution operations; S402: Calculate the three-dimensional coordinate difference and feature difference between each nearest point in the nearest point set and the key point, and obtain the micro-relative position features through multilayer perceptron encoding; S403: The macroscopic contour features and the microscopic relative position features are spliced ​​together, the feature gating map is generated by a gating activation function, and the feature gating map is multiplied element-wise with the high-dimensional local features to obtain the key point fusion features.

8. The three-dimensional target detection method based on lidar point clouds according to claim 7, characterized in that, Specifically, S402 includes: S4021: Calculate the coordinate difference vector between the three-dimensional spatial coordinates of each nearest point in the nearest point set and the three-dimensional spatial coordinates of the key point, and the feature difference vector between the feature of each nearest point in the nearest point set and the feature of the key point; S4022: Concatenate the coordinate difference vector with the feature difference vector, input the concatenation vector into a multilayer perceptron for nonlinear transformation, and obtain the micro-relative position sub-features corresponding to each nearest neighbor point; S4023: Perform pooling operation on the micro-relative position sub-features of all nearest neighbor points to obtain the micro-relative position features.

9. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S5 specifically includes: S501: Input the two-dimensional projection features into the region proposal network to generate multiple initial three-dimensional candidate boxes; S502: Determine the three-dimensional spatial candidate region corresponding to each initial three-dimensional candidate box, and use the region of interest pooling operator to aggregate the key point fusion features into the three-dimensional spatial candidate region inside the initial three-dimensional candidate box to obtain the filled three-dimensional spatial candidate region features.

10. The three-dimensional target detection method based on lidar point clouds according to claim 1, characterized in that, S6 specifically includes: S601: Input the filled 3D spatial candidate region features into the detection head, and output the target category probability and 3D bounding box regression parameters; S602: Calculate the target center point offset, target size parameters, and target yaw angle based on the three-dimensional bounding box regression parameters, and generate a predicted three-dimensional bounding box set; S603: Calculate the intersection-union ratio (IUGR) between each predicted 3D bounding box in the predicted 3D bounding box set, remove redundant boxes by non-maximum suppression, and output the 3D target detection result.

Images