A 3D target detection method, system, device, and medium

Abstract

This invention relates to the field of point cloud image analysis technology, specifically to a 3D target detection method, system, device, and medium. First, the points in the target point cloud data are voxelized and meshed to obtain a meshed voxelized point cloud. Then, foreground points in the meshed voxelized point cloud are selected, and after feature dimension transformation, several differential attention processes, downsampling processes, and foreground point update labeling processes are performed to obtain multi-scale foreground point cloud features. Next, based on the multi-scale foreground point cloud features, the region of interest is locked and key points are sampled. Features from different ranges of neighborhoods are aggregated, and multi-dimensional features are fused through channel stitching. Then, the features of the region of interest are aggregated and stitched to obtain the 3D target detection features. Finally, the 3D target detection features are processed by target detection to obtain the detection result. Applied to the field of 3D target detection based on point cloud image analysis, this method is highly efficient and robust, improving detection accuracy and enhancing adaptability to complex environments.

Description

Technical Field

[0001] This invention relates to the field of point cloud image analysis technology, specifically to a 3D target detection method, system, device, and medium. Background Technology

[0002] 3D object detection is a crucial computer vision task that aims to analyze and locate objects in space from 3D point cloud image data. Compared to traditional image analysis, point cloud-based 3D object detection can directly utilize the spatial geometric information of point clouds, resulting in stronger distance estimation and stereo structure analysis capabilities.

[0003] However, existing 3D object detection methods still face many challenges. First, point cloud data is sparse and uneven, especially in distant areas or severely occluded scenes, where analysis performance tends to deteriorate. Furthermore, models may still experience omissions, inaccurate localization, and classification errors when dealing with complex backgrounds, targets of different scales, and diverse scenes. Summary of the Invention

[0004] The purpose of this invention is to provide a 3D target detection method, system, device, and medium.

[0005] The technical solution of this invention is as follows:

[0006] A 3D target detection method includes the following operations:

[0007] S1. Obtain the target point cloud data, perform voxelization, and map it to grid coordinates to obtain the grid voxelized point cloud;

[0008] S2. Obtain the mean value of the point cloud features of each voxel in the gridded voxel point cloud, arrange them in descending order, and take the voxel center corresponding to the mean value of the first k points as the foreground points. Mark them in the gridded voxel point cloud data to obtain the marked voxel point cloud. Transform the feature dimension of the marked voxel point cloud to obtain the marked voxel point cloud features. The marked voxel point cloud features are processed through several differential attention processes, downsampling processes, and foreground point update labeling processes. Extract the output of the downsampling process in the intermediate process and the output of the final differential attention process to obtain the multi-scale foreground point cloud features.

[0009] S3. Based on multi-scale foreground point cloud features, obtain the region of interest (ROI) of the mesh voxelized point cloud. After sampling, obtain key points. Aggregate the different range neighborhood point cloud sets corresponding to each key point in the marked voxelized point cloud features and multi-scale foreground point cloud features to obtain key point neighborhood aggregation features at several scales. Stitch along the channel dimension to obtain key point channel aggregation features. Based on the key point channel aggregation features, after feature aggregation processing of the ROI of the mesh voxelized point cloud, obtain the 3D target detection features through feature stitching.

[0010] The S4 and 3D target features are processed by target detection to obtain the 3D target detection results.

[0011] S2's differential attention processing targets the foreground point. The differential attention processing operation is as follows: Step 1: Based on the spiral receptive field method, obtain several neighbor points of the foreground point, and subtract them from the position of the foreground point to obtain several relative positions of the neighbors. These relative positions are then encoded to obtain several relative position features of the neighbors. Step 2: Based on the features of the foreground point, the relative position features of each neighbor point, and the features of each neighbor point, obtain the foreground point query features, the key features of the foreground point and each neighbor point, and the value features. The key features and value features of the foreground point and each neighbor point form the key feature matrix and the value feature matrix, respectively. Step 3: Double the attention head of the foreground query features, key feature matrix, and value feature matrix to obtain the foreground query doubled features, doubled key feature matrix, and doubled value feature matrix; Step 4: Obtain the attention doubling weight based on the foreground query doubled features and doubled key feature matrix; The attention doubling weight is then processed by differential weight adjustment to obtain the attention differential weight; Step 5: Obtain the initial differential attention features based on the attention differential weight and doubled value feature matrix; The initial differential attention features are then processed by residual connections and a feedforward neural network to obtain the foreground differential attention features, which are used to perform downsampling operations.

[0012] Neighbor points are points on a spiral line obtained with a fixed step size, fixed curvature, and fixed number of turns, centered on previous attractions.

[0013] The foreground point difference attention feature is calculated using the following formula:

[0014] ,

[0015] ,

[0016] For the first i Normalized differential attention features for each foreground point For the first i The characteristics of the scenic spot This is handled by the Dropout function. For layer normalization processing, For processing by feedforward neural networks, For the first i Foreground attention features of each foreground location.

[0017] The method for obtaining the region of interest in the meshed voxelized point cloud in S3 is as follows: the foreground point cloud feature corresponding to the output of the final differential attention processing in S2 is used as the target foreground point cloud feature; the target foreground point cloud feature is converted into a heat map, and the region corresponding to the heat value greater than the heat value threshold in the heat map is used as the key region; the corresponding position of the key region in the meshed voxelized point cloud is used as the region of interest.

[0018] The method for obtaining keypoint neighborhood aggregation features is achieved through the following formula:

[0019] ,

[0020] ,

[0021] The first key point neighborhood aggregation feature v The first key point e Neighborhood aggregation features within a range, For the first v Key features of each key point For the first The first key point is the center e The neighborhood region corresponding to the range of the first range is the first m One neighboring point, for and Neighbor weights between For the first The total number of neighboring points within the neighborhood region corresponding to the e-th range centered on a key point, where E is the total number of ranges. The first key point neighborhood aggregation feature v Neighborhood aggregation features of key points.

[0022] In S3, feature aggregation is implemented using the following formula:

[0023] ,

[0024] For the first r Aggregated features of regions of interest For the first r Key features of a region of interest For the first r Key points in regions of interest With point The weights between them For the first r The first region of interest n One point, For the first r The total number of points in each region of interest.

[0025] A 3D target detection system for implementing the above-mentioned 3D target detection method includes:

[0026] The mesh voxelized point cloud generation module is used to acquire target point cloud data, voxelize it, and map it to mesh coordinates to obtain a mesh voxelized point cloud.

[0027] The multi-scale foreground point cloud feature generation module is used to obtain the mean point cloud feature of each voxel in the gridded voxelized point cloud, arrange them in descending order, and take the voxel center corresponding to the mean point cloud feature of the first k points as the foreground point, and mark them in the gridded voxelized point cloud data to obtain the marked voxelized point cloud; the marked voxelized point cloud is transformed in feature dimension to obtain the marked voxelized point cloud feature; the marked voxelized point cloud feature is processed by several differential attention processing, downsampling processing and foreground point update label processing, and the output of the downsampling processing in the intermediate process and the output of the final differential attention processing are extracted to obtain the multi-scale foreground point cloud feature;

[0028] The 3D target detection feature generation module is used to obtain the region of interest (ROI) of the mesh voxelized point cloud based on multi-scale foreground point cloud features, and obtain key points through sampling processing. Then, it aggregates the different range neighborhood point cloud sets corresponding to each key point in both the marked voxelized point cloud features and the multi-scale foreground point cloud features to obtain key point neighborhood aggregated features at several scales. These aggregated features are then stitched along the channel dimension to obtain key point channel aggregated features. Based on the key point channel aggregated features, the ROI of the mesh voxelized point cloud is further aggregated and stitched to obtain the 3D target detection features.

[0029] The 3D target detection result generation module is used to obtain 3D target detection results by processing the 3D target features to be detected.

[0030] A 3D target detection device includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described 3D target detection method.

[0031] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described 3D target detection method.

[0032] The beneficial effects of this invention are as follows:

[0033] This invention provides a 3D target detection method. First, the points in the target point cloud data are voxelized, mapping the point coordinates to grid coordinates. Points with the same grid coordinates are grouped into the same voxel, resulting in a grid-voxelized point cloud. Then, based on the mean point cloud feature value of each voxel in the grid-voxelized point cloud, foreground points are selected, focusing on key information to reduce redundancy. The foreground points are then marked in the grid-voxelized point cloud data. After feature dimension transformation, several differential attention processes, downsampling processes, and foreground point update marking processes are performed. This enhances feature correlation while preserving the core features of the foreground points, and integrates details and global information at different levels, thereby effectively enriching the feature information of the foreground points. The downsampling output from the intermediate processes is extracted, as well as the most... The final attention output yields multi-scale foreground point cloud features. Next, based on these features, the region of interest (ROI) is located and key points are sampled. Multi-scale contextual information is captured by aggregating neighborhood features from different ranges. Multi-dimensional features are then fused through channel stitching. Further aggregation and stitching of ROI features preserves both local details and global relationships of key points while incorporating rich information from multiple scales and channels. This makes the 3D target detection features more comprehensive and discriminative, effectively improving the representation ability of 3D target features. Finally, the 3D target detection features undergo target detection processing to obtain 3D target detection results. Applied to the field of 3D target detection based on point cloud image analysis, this method demonstrates high efficiency and robustness, improving detection accuracy and enhancing adaptability to complex environments. Attached Figure Description

[0034] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0035] In the attached diagram:

[0036] Figure 1 This is a flowchart illustrating the method of this embodiment.

[0037] Figure 2 The image shown is a 3D target detection result diagram of the method in this embodiment. Detailed Implementation

[0038] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings.

[0039] This embodiment provides a 3D target detection method; see [link to documentation]. Figure 1 This includes the following operations:

[0040] S1. Obtain the target point cloud data, perform voxelization, and map it to grid coordinates to obtain the grid voxelized point cloud;

[0041] S2. Obtain the mean value of the point cloud features of each voxel in the gridded voxel point cloud, arrange them in descending order, and take the voxel center corresponding to the mean value of the first k points as the foreground points. Mark them in the gridded voxel point cloud data to obtain the marked voxel point cloud. Transform the feature dimension of the marked voxel point cloud to obtain the marked voxel point cloud features. The marked voxel point cloud features are processed through several differential attention processes, downsampling processes, and foreground point update labeling processes. Extract the output of the downsampling process in the intermediate process and the output of the final differential attention process to obtain the multi-scale foreground point cloud features.

[0042] S3. Based on multi-scale foreground point cloud features, obtain the region of interest (ROI) of the mesh voxelized point cloud. After sampling, obtain key points. Aggregate the different range neighborhood point cloud sets corresponding to each key point in the marked voxelized point cloud features and multi-scale foreground point cloud features to obtain key point neighborhood aggregation features at several scales. Stitch along the channel dimension to obtain key point channel aggregation features. Based on the key point channel aggregation features, after feature aggregation processing of the ROI of the mesh voxelized point cloud, obtain the 3D target detection features through feature stitching.

[0043] The S4 and 3D target features are processed by target detection to obtain the 3D target detection results.

[0044] The specific operating steps are detailed below.

[0045] S1. Obtain the target point cloud data, perform voxelization, and map it to grid coordinates to obtain the grid voxelized point cloud.

[0046] Acquire LiDAR point cloud data of the target, voxelize the points in the target point cloud data, map the coordinates of the points to grid coordinates, and group points with the same grid coordinates into the same voxel to obtain a grid voxelized point cloud. This can reduce the amount of data and unify the spatial scale, which can accelerate subsequent calculations and facilitate more accurate selection of foreground points.

[0047] S2. Obtain the mean point cloud feature of each voxel in the gridded voxelized point cloud, arrange them in descending order, and take the voxel center corresponding to the mean point cloud feature of the first k points as the foreground points. Mark them in the gridded voxelized point cloud data to obtain the marked voxelized point cloud. Transform the feature dimension of the marked voxelized point cloud to obtain the marked voxelized point cloud features. The marked voxelized point cloud features are processed through several differential attention processes, downsampling processes, and foreground point update marking processes. Extract the downsampling output in the intermediate process and the final attention output to obtain the multi-scale foreground point cloud features.

[0048] Based on the mean point cloud features of each voxel in the gridded voxelized point cloud, foreground points are selected, focusing on key information to reduce redundancy. Then, the foreground points are marked in the gridded voxelized point cloud data. After feature dimension transformation, several differential attention processing, downsampling processing, and foreground point update and labeling processing are performed. This enhances feature correlation while retaining the core features of the foreground points and integrating details and global information at different levels, thereby effectively enriching the feature information of the foreground points. The downsampling output in the intermediate process and the final attention output are extracted to obtain multi-scale foreground point cloud features.

[0049] First, the mean value of the point cloud features of each voxel in the gridded voxelized point cloud is obtained and arranged in descending order. The voxel centers corresponding to the mean values ​​of the top k voxel features are used as foreground points and are marked in the gridded voxelized point cloud data to obtain the marked voxelized point cloud.

[0050] Then, the labeled voxelized point cloud undergoes feature dimension transformation (including but not limited to linear layer mapping) to obtain labeled voxelized point cloud features. Feature dimension transformation is performed, but not limited to linear layer mapping, and is preferably achieved using the ReLU activation function.

[0051] Next, the labeled voxelized point cloud features are processed through several differential attention processes, downsampling processes, and foreground point update labeling processes. The output of the downsampling process in the intermediate process and the output of the final differential attention process are extracted to obtain multi-scale foreground point cloud features.

[0052] Specifically, foreground points in the voxelized point cloud features are labeled, and then processed by differential attention and downsampling to extract the first-scale foreground point cloud features. After updating the foreground points of the first-scale foreground point cloud features based on the mean of the point cloud features, differential attention and downsampling are performed to extract the second-scale foreground point cloud features. After updating the second-scale foreground point cloud features based on the mean of the point cloud features, differential attention is performed as the final attention output to obtain the third-scale foreground point cloud features.

[0053] The differential attention processing targets the foreground points containing the main information, and the operation steps of differential attention processing are as follows.

[0054] Step 1: Based on the spiral receptive field method, obtain several neighboring points of the (current) foreground point, and subtract them from the position of the foreground point to obtain several relative positions of the neighbors. After encoding, obtain several relative position features of the neighbors.

[0055] The neighbor points of the foreground point are the points on the spiral line obtained with a fixed step size, fixed curvature, and fixed number of turns, centered on the previous point. This method can preserve more local and global information while consuming the same amount of computing resources.

[0056] Step 2: Based on the features of the foreground point, the relative position features of the foreground point with each neighbor point, and the features of each neighbor point, obtain the query features of the foreground point, the key features of the foreground point and each neighbor point, and the value features; the key features and value features of the foreground point and each neighbor point form the key feature matrix and the value feature matrix, respectively.

[0057] No. i The features of previous attractions in the search, the first i The first scenic spot and the first j The key features and value features of each neighbor can be calculated using the following formula:

[0058] ,

[0059] ,

[0060] ,

[0061] ,

[0062] ,

[0063] ,

[0064] For the first i The first scenic spot and the first j The relative positions of the neighbors of each neighboring point , The first i Each scenic spot , No. j The voxel coordinates of the neighboring points For the first i The first scenic spot and the first j The relative positional characteristics of neighboring points, This is a one-dimensional convolution process. It is the ReLU activation function. For encoding weights, For the first i Location coding features of each foreground point For the first i Query features for each foreground point To query weight, For the first i Each foreground feature (vector) For the first i The first scenic spot and the first j Key features of neighboring points, For the first j Features (vectors) of each neighboring point. For key weights, For the first i The first scenic spot and the first j The value characteristics of each neighboring point Value weights.

[0065] Step 3: Double the attention heads of the foreground query features, key feature matrix, and value feature matrix to obtain doubled foreground query features, doubled key feature matrix, and doubled value feature matrix. Specifically, double the number of attention heads for the foreground query features and key feature matrix, and double the attention head dimension of each value feature in the value feature matrix. The doubled query features and key features are equivalent to each head having two channels, while doubling the dimension of the value features is to accommodate the dimensions of two channels, resulting in doubled foreground query features, doubled key feature matrix, and doubled value feature matrix.

[0066] No. i The features of previous attractions in the search, the first i The first scenic spot and the first j The key and value features of each neighbor point, after doubling the number of attention heads, can be achieved using the following formula:

[0067] ,

[0068] ,

[0069] ,

[0070] For the first i The feature of doubling the search volume for previous attractions. For the first i The first scenic spot and the first j The doubling key feature of neighboring points For the first i The first scenic spot and the first j The doubling value feature of neighboring points, , The sizes of the first and second batches are respectively. , Query the number of heads, the number of keys, and the number of values, respectively. d The number of attention heads is doubled. The original 4-scale attention heads can be doubled to 8 sub-heads. The attention head dimensions of query features, key features, and value features of different channels are doubled in order to match the dimensions of the two channels.

[0071] Step 4: Based on the doubling feature and doubling key feature matrix of the foreground points, obtain the attention doubling weight; the attention doubling weight is processed by differential weight adjustment to eliminate the noise generated by the standard attention and obtain the attention differential weight.

[0072] The doubling attention weight includes two sets of channel doubling attention weights, which can be calculated using the following formula:

[0073] ,

[0074] For the first i The first scenic spot b Double the attention weight of the group channel (b=1, 2). For the first i The first scenic spot b Features of the search for attractions in the group channel ( = ), For the first i The first scenic spot b Key feature matrix of group channels ( = ), For attention masking, d This represents the number of attention head dimensions.

[0075] Attention differential weights are calculated using the following formula:

[0076] ,

[0077] ,

[0078] ,

[0079] ,

[0080] ,

[0081] For the first i Attention difference weights for each foreground point For the first i The first group of channels in the doubled attention weighting of the foreground attractions doubled attention weighting. This is the differential weighting adjustment factor. For the first i The second group of channels in the doubled attention weighting of the first-viewpoint doubled attention weighting This is the first weighted adjustment coefficient. This is the second weighting adjustment coefficient. This is the initial weight adjustment coefficient. , These are the first and second elements of the query feature (vector) processed by the attention mechanism. o The first and second weight adjustment factors for each dimension , These are the first and second key features (vectors) processed by the attention mechanism. o The first and second weight adjustment factors for each dimension O This represents the total number of vector dimensions. For depth number, , , , It is a learnable variable and can be initialized using a normal distribution.

[0082] Step 5: Based on the attention difference weights and doubling value feature matrix, obtain the initial difference attention features; the initial difference attention features are processed by residual connections and feedforward neural networks to obtain the (current) foreground difference attention features, which are used to perform downsampling operations.

[0083] No. i The initial differential attention features for each foreground point are calculated using the following formula:

[0084] ,

[0085] ,

[0086] ,

[0087] ,

[0088] For the first i Initial differential attention features for each foreground point For the first i The doubling value feature matrix of each foreground point For normalization function, The first optimized differential attention feature, , , These are the first batch size, the number of key headers, and the number of attention head dimensions, respectively. For the second optimized differential attention feature, For the first i Attention output for each foreground point Output weights for differential attention.

[0089] No. iThe foreground attention features of each foreground point are calculated using the following formula:

[0090] ,

[0091] ,

[0092] For the i-th foreground point, the normalized differential attention feature is... For the first i Feature vectors of each foreground point This is handled by the Dropout function. For layer normalization processing, For processing by feedforward neural networks, For the first i Foreground attention features of each foreground location.

[0093] The downsampling process mainly involves downsampling the coordinates of the foreground point in the foreground point differential attention feature, which is calculated using the following formula:

[0094] ,

[0095] For the first i The downsampled coordinates of the foreground points For the first i Coordinate features in the foreground difference attention features of each foreground point For downsampling step size, voxel size, and The new voxel size is obtained after multiplication. l This is the downsampling offset.

[0096] S3. Based on multi-scale foreground point cloud features, obtain the region of interest (ROI) of the mesh voxelized point cloud. After sampling, obtain key points. Aggregate the different range neighborhood point cloud sets corresponding to each key point in the marked voxelized point cloud features and multi-scale foreground point cloud features to obtain key point neighborhood aggregation features at several scales. Stitch along the channel dimension to obtain key point channel aggregation features. Based on the key point channel aggregation features, after feature aggregation processing of the ROI of the mesh voxelized point cloud, obtain the 3D target detection features through feature stitching.

[0097] Based on multi-scale foreground cloud features, the region of interest is located and key points are sampled. By aggregating neighborhood features of different ranges, multi-scale contextual information is captured. Multi-dimensional features are then fused through channel splicing. Finally, the features of the region of interest are aggregated and spliced. This process not only preserves the local details and global correlations of key points, but also integrates rich information from multiple scales and channels. This makes the 3D target detection features more comprehensive and discriminative, effectively improving the ability to represent 3D target features and providing more accurate and informative feature support for subsequent detection tasks.

[0098] First, based on the multi-scale foreground point cloud features, the region of interest of the grid voxelized point cloud is obtained, and after sampling processing (which can be achieved by the SPC sampling method based on the sector region), the key points are obtained.

[0099] The method for obtaining the region of interest in the mesh voxelized point cloud is as follows: the foreground point cloud feature (third-scale foreground point cloud feature) corresponding to the final attention output in S2 is used as the target foreground point cloud feature; the target foreground point cloud feature is converted into a heat map, and the regions in the heat map whose heat values ​​are greater than the heat value threshold (the point cloud features of these regions are obvious) are key regions; the corresponding positions of the key regions in the mesh voxelized point cloud are used as the regions of interest.

[0100] Then, the different range neighborhood point cloud sets of each key point in the marked voxelized point cloud features and multi-scale foreground point cloud features (inclusive) are aggregated to obtain key point neighborhood aggregated features at several scales.

[0101] Using labeled voxelized point cloud features as the current scale, the keypoint neighborhood aggregation feature is obtained through the following formula:

[0102] ,

[0103] ,

[0104] The first key point neighborhood aggregation feature v The first key point e Neighborhood aggregation features within a range, For the first v Key feature vectors of key points. For the first The neighborhood region corresponding to the e-th range centered on the key point. m One neighboring point, for and Neighbor weights between For the first The total number of neighboring points within the neighborhood region corresponding to the e-th range centered on a key point, where E is the total number of ranges. The first key point neighborhood aggregation feature v Neighborhood aggregation features of key points.

[0105] Next, the keypoint neighborhood aggregation features corresponding to the marked voxelized point cloud features and the multi-scale foreground point cloud features (including the first-scale foreground point cloud features, the second-scale foreground point cloud features, and the third-scale foreground point cloud features) are spliced ​​along the channel dimension to obtain the keypoint channel aggregation features.

[0106] Finally, based on the key point channel aggregation features, the region of interest of the mesh voxelized point cloud is subjected to feature aggregation processing, and then the features are stitched together to obtain the 3D target detection features.

[0107] Feature aggregation is achieved through the following formula:

[0108] ,

[0109] For the first r Aggregated features of regions of interest For the first r Key point features (vectors) of a region of interest. For the first r Key points in regions of interest With point The weights between them are based on key points. With point The distance is obtained from the Gaussian formula. For the first r The first region of interest n One point, For the first r The total number of points in each region of interest.

[0110] S4 and 3D target features are processed by target detection to obtain 3D target detection results, which then participate in... Figure 2 .

[0111] This embodiment also provides a 3D target detection system for implementing the above-described 3D target detection method, including:

[0112] The mesh voxelized point cloud generation module is used to acquire target point cloud data, voxelize it, and map it to mesh coordinates to obtain a mesh voxelized point cloud.

[0113] The multi-scale foreground point cloud feature generation module is used to obtain the mean point cloud feature of each voxel in the gridded voxelized point cloud, arrange them in descending order, and take the voxel center corresponding to the mean point cloud feature of the first k points as the foreground point, and mark them in the gridded voxelized point cloud data to obtain the marked voxelized point cloud; the marked voxelized point cloud is transformed in feature dimension to obtain the marked voxelized point cloud feature; the marked voxelized point cloud feature is processed by several differential attention processing, downsampling processing and foreground point update label processing, and the output of the downsampling processing in the intermediate process and the output of the final differential attention processing are extracted to obtain the multi-scale foreground point cloud feature;

[0114] The 3D target detection feature generation module is used to obtain the region of interest (ROI) of the mesh voxelized point cloud based on multi-scale foreground point cloud features, and obtain key points through sampling processing. Then, it aggregates the different range neighborhood point cloud sets corresponding to each key point in both the marked voxelized point cloud features and the multi-scale foreground point cloud features to obtain key point neighborhood aggregated features at several scales. These aggregated features are then stitched along the channel dimension to obtain key point channel aggregated features. Based on the key point channel aggregated features, the ROI of the mesh voxelized point cloud is further aggregated and stitched to obtain the 3D target detection features.

[0115] The 3D target detection result generation module is used to obtain 3D target detection results by processing the 3D target features to be detected.

[0116] This embodiment also provides a 3D target detection device, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described 3D target detection method.

[0117] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described 3D target detection method.

[0118] This embodiment provides a 3D target detection method. First, the points in the target point cloud data are voxelized, mapping the point coordinates to grid coordinates. Points with the same grid coordinates are grouped into the same voxel, resulting in a grid-voxelized point cloud. Then, based on the mean point cloud feature value of each voxel in the grid-voxelized point cloud, foreground points are selected, focusing on key information to reduce redundancy. The foreground points are then marked in the grid-voxelized point cloud data. After feature dimension transformation, several differential attention processes, downsampling processes, and foreground point update marking processes are performed. This enhances feature correlation while preserving the core features of the foreground points, and integrates details and global information at different levels, effectively enriching the feature information of the foreground points. The downsampling output from the intermediate processes is extracted. Finally, attention output yields multi-scale foreground point cloud features. Next, based on these features, the region of interest (ROI) is located and key points are sampled. Multi-scale contextual information is captured by aggregating neighborhood features from different ranges. Multi-dimensional features are then fused through channel stitching. Further aggregation and stitching of ROI features preserves both local details and global relationships of key points while integrating rich multi-scale and multi-channel information, making the 3D target detection features more comprehensive and discriminative, effectively improving the representation ability of 3D target features. Finally, the 3D target detection features undergo target detection processing to obtain 3D target detection results. Applied to the field of 3D target detection based on point cloud image analysis, this method is efficient and robust, improving detection accuracy and enhancing adaptability to complex environments.

Claims

1. A 3D target detection method, characterized in that, This includes the following operations: S1. Obtain the target point cloud data, perform voxelization, and map it to grid coordinates to obtain the grid voxelized point cloud; S2. Obtain the mean point cloud feature of each voxel in the mesh voxelized point cloud, arrange them in descending order, take the voxel center corresponding to the mean point cloud feature of the first k points as the foreground point, and mark them in the mesh voxelized point cloud data to obtain the marked voxelized point cloud. The labeled voxelized point cloud is transformed in terms of feature dimensions to obtain the labeled voxelized point cloud features. The labeled voxelized point cloud features are processed through three differential attention processes, downsampling processes, and foreground point update labeling processes. The output of the downsampling process in the intermediate process and the output of the final differential attention process are extracted to obtain multi-scale foreground point cloud features. Differential attention processing targets foreground points, and the operation of differential attention processing is as follows: Step 1: Based on the spiral receptive field method, obtain several neighboring points of the foreground point, and calculate the difference between each of them and the position of the foreground point to obtain several relative positions of the neighbors. Then, encode each of them to obtain several relative position features of the neighbors. Neighbor points are points on a spiral line obtained with a fixed step size, fixed curvature, and fixed number of turns, centered on previous attractions. Step 2: Based on the features of the foreground point, the relative position features of the foreground point with each neighbor point, and the features of each neighbor point, obtain the query features of the foreground point, the key features of the foreground point and each neighbor point, and the value features. The key features and value features of the foreground point and each of its neighboring points form the key feature matrix and the value feature matrix, respectively. Step 3: Double the attention of the foreground query features, key feature matrix and value feature matrix to obtain the foreground query doubled features, doubled key feature matrix and doubled value feature matrix; Step 4: Based on the doubling feature and doubling key feature matrix of the foreground points, obtain the attention doubling weight; The attention doubling weight is then adjusted using differential weighting to obtain the attention differential weight; Step 5: Based on the attention difference weights and the doubling value feature matrix, obtain the initial difference attention features; The initial differential attention features are processed by residual connections and a feedforward neural network to obtain the foreground differential attention features, which are used to perform downsampling operations; S3. Based on the multi-scale foreground point cloud features, obtain the region of interest (ROI) of the mesh voxelized point cloud, and obtain the key points after sampling processing. The method for obtaining the ROI is to take the foreground point cloud features corresponding to the output of the final differential attention processing in S2 as the target foreground point cloud features. The cloud features of the target foreground are converted into a heat map. The areas in the heat map with heat values ​​greater than the heat value threshold are designated as key areas. The key regions are located at their corresponding positions in the meshed voxelized point cloud, which are then designated as regions of interest. The different range neighborhood point cloud sets corresponding to each key point in the marked voxelized point cloud features and multi-scale foreground point cloud features are aggregated to obtain key point neighborhood aggregated features of several scales. These features are then stitched together along the channel dimension to obtain key point channel aggregated features. Based on key point channel aggregation features, the region of interest of the mesh voxelized point cloud is subjected to feature aggregation processing, and then the features are stitched together to obtain the 3D target detection features. The S4 and 3D target features are processed by target detection to obtain the 3D target detection results.

2. The 3D target detection method according to claim 1, characterized in that, The foreground point difference attention feature is calculated using the following formula: ， ， For the first i Normalized differential attention features for each foreground point For the first i The characteristics of the scenic spot This is handled by the Dropout function. For layer normalization processing, For processing by feedforward neural networks, For the first i Foreground attention features of each foreground location For the first i Attention output for each foreground point.

3. The 3D target detection method according to claim 1, characterized in that, In S3, the method for obtaining keypoint neighborhood aggregation features is achieved through the following formula: ， ， The first key point neighborhood aggregation feature v The first key point e Neighborhood aggregation features within a range, For the first v Key features of each key point For the first The first key point is the center e The neighborhood region corresponding to the range is the first m One neighboring point, for and Neighbor weights between For the first The total number of neighboring points within the neighborhood region corresponding to the e-th range centered on a key point, where E is the total number of ranges. The first key point neighborhood aggregation feature v Neighborhood aggregation features of key points.

4. The 3D target detection method according to claim 1, characterized in that, In S3, feature aggregation is implemented using the following formula: ， For the first r Aggregated features of regions of interest For the first r Key features of a region of interest For the first r Key points in regions of interest With point The weights between them For the first r The first region of interest n One point, For the first r The total number of points in each region of interest.

5. A 3D target detection system, characterized in that, The method for implementing the 3D target detection method of claim 1 is characterized by comprising: The mesh voxelized point cloud generation module is used to acquire target point cloud data, voxelize it, and map it to mesh coordinates to obtain a mesh voxelized point cloud. The multi-scale foreground point cloud feature generation module is used to obtain the mean point cloud feature of each voxel in the gridded voxelized point cloud, arrange them in descending order, and take the voxel center corresponding to the mean point cloud feature of the first k points as the foreground point, and mark them in the gridded voxelized point cloud data to obtain the marked voxelized point cloud; the marked voxelized point cloud is transformed in feature dimension to obtain the marked voxelized point cloud feature; the marked voxelized point cloud feature is processed by several differential attention processing, downsampling processing and foreground point update label processing, and the output of the downsampling processing in the intermediate process and the output of the final differential attention processing are extracted to obtain the multi-scale foreground point cloud feature; The 3D target detection feature generation module is used to obtain the region of interest (ROI) of the mesh voxelized point cloud based on multi-scale foreground point cloud features, and obtain key points through sampling processing. Then, it aggregates the different range neighborhood point cloud sets corresponding to each key point in both the marked voxelized point cloud features and the multi-scale foreground point cloud features to obtain key point neighborhood aggregated features at several scales. These aggregated features are then stitched along the channel dimension to obtain key point channel aggregated features. Based on the key point channel aggregated features, the ROI of the mesh voxelized point cloud is further aggregated and stitched to obtain the 3D target detection features. The 3D target detection result generation module is used to obtain 3D target detection results by processing the 3D target features to be detected.

6. A 3D target detection device, characterized in that, It includes a processor and a memory, wherein the processor implements the 3D target detection method as described in any one of claims 1-4 when executing a computer program stored in the memory.

7. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the 3D target detection method as described in any one of claims 1-4.

Images