Outdoor three-dimensional target detection method based on target center and class perception

Abstract

The invention discloses an outdoor three-dimensional target detection method based on a target center and class perception, and the method specifically comprises the following steps: obtaining a three-dimensional point cloud data set D, and dividing the data set D into a training set Dtrain and a test set Dtest; constructing an outdoor three-dimensional target detection network structure Net based on a target center and class perception; defining a loss function; training parameters are set, the training set Dtrain is input into an outdoor three-dimensional target detection network structure Net for training, and a network model Modeltrain is obtained; performing target detection on the outdoor three-dimensional target detection network structure Net on the test set Dtest by using the network model Modeltrain, and outputting a detection result Predtest; and the detection performance of the network model Modeltrain is evaluated according to the detection result Predtest. According to the invention, the precision of three-dimensional target detection can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an outdoor three-dimensional target detection method based on target center and class perception. Background Art

[0002] Outdoor 3D target detection is based on point clouds, images and other information collected by 2D or 3D sensors, and uses the designed detection algorithm to extract and analyze scene information, so as to achieve the purpose of perceiving objects in space. It is widely used in many fields such as autonomous driving and intelligent robots, and has important theoretical and application value. The 3D laser point cloud data collected by LiDAR sensors are often affected by complex environments, occlusions and other factors, which makes the collected object point cloud low in integrity and the target and background mixed, making detection more difficult. The existing 3D detection methods have low detection accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide an outdoor three-dimensional target detection method based on target center and class perception, which has the characteristic of improving the accuracy of three-dimensional target detection.

[0004] The technical solution adopted by the present invention is an outdoor three-dimensional target detection method based on target center and class perception, which specifically includes the following steps:

[0005] Step 1: Obtain a 3D point cloud dataset D and divide it into a training set D train and the test set D test ;

[0006] Step 2: Construct an outdoor 3D target detection network structure Net based on target center and class perception;

[0007] Step 3: Define the loss function;

[0008] Step 4: Set the training parameters and set the training set D train Input the outdoor 3D target detection network structure Net for training and obtain the network model Model train ;

[0009] Step 5: Use the network model Model to detect the outdoor three-dimensional target network structure Net train In the test set D test Perform target detection on it and output the detection result Pred test ;

[0010] Step 6: According to the detection result Pred test Network Model train The detection performance was evaluated.

[0011] The present invention is also characterized in that:

[0012] The specific process of step 1 is:

[0013] Download the 3D point cloud dataset D from the official website of the public dataset, D = {Pt, Gt}, where Pt represents the scene point cloud set and Gt represents the true value dataset. Divide the dataset D into the training set D according to the ratio of A:B train = {Pt train ,Gt train} and test set D test = {Pt test ,Gt test},in, represents the point cloud of the ath scene in the training set, represents the target truth value of the ath scene in the training set,

[0014] represents the point cloud of the bth scene in the test set, represents the target true value of the b-th scene in the test set, 1≤a≤A, 1≤b≤B, A is the total number of scenes in the training set, and B is the total number of scenes in the test set.

[0015] The specific process of step 2 is:

[0016] Step 2.1: Construct a point cloud voxelization module to convert the point cloud of the ath scene in the training set into Perform point cloud voxelization to obtain the original voxel dataset V 0 ;

[0017] Step 2.2: Construct a voxel feature extraction module and convert the original voxel dataset V 0 Input into the sparse 3D CNN network for multi-scale downsampling voxel feature extraction, and obtain L layers of sparse voxel datasets V with different downsampling scales. Net = {V l ,…,V l ,…,V L},in, V l is the sparse voxel dataset of the lth layer, 1≤l≤L, L is the total number of downsampling layers, Ml represents the number of non-empty voxels in the lth layer, and sparse voxels Expressed as Represents sparse voxels The feature vector of l , D l ∈T, Represents sparse voxels The index number in the x, y and z directions is expressed as

[0018] Step 2.3: Build the target center point cloud generation module and transform the L-th layer sparse voxel dataset V L Perform voting and filling to obtain an expanded sparse voxel dataset and the corresponding sparse voxel enhanced features in, V Ex represents the augmented virtual target center voxel dataset;

[0019] Step 2.4: Construct a proposal box generation module to enhance the sparse voxel features As input, the function view() in the Pytorch library is used to enhance the features of sparse voxels. Convert to bird's eye view feature map Bird's-eye view feature map Input into the region proposal network under the open source Openpcdet library to obtain a set of 3D proposal boxes Where 1≤n p ≤N p , N p Represents the total number of 3D proposal boxes, Indicates a proposal box The center coordinates of Represents the proposal frame The height, width and length of Indicates a proposal box The z-axis rotation angle, Indicates a proposal box The category number, Indicates a proposal box Category score of

[0020] Step 2.5: Construct a target class perception feature enhancement module to perform Net = {V 1 ,…,V l ,…,V L}Sampling and clustering are performed to obtain a class-aware enhanced sparse voxel dataset

[0021] Step 2.6: Construct the target area grid pooling module to enhance the class-aware sparse voxel dataset Gridding and sampling with the three-dimensional proposal box set PBox to obtain the proposal box feature set F PBox ;

[0022] Step 2.7: Construct the target refinement module and transform the proposed box feature set F pBox Perform target refinement prediction to obtain the refined proposal box BBox and confidence Conf, where:

[0023] Represents the category score of the proposal box after target refinement, 1≤n b ≤N b , N b Represents the total number of proposal boxes after target refinement, Represents the center coordinates of the proposal box after target refinement, Respectively represent the height, width, and length of the proposed box after target refinement, Indicates the z-axis rotation angle of the proposal box.

[0024] The specific process of step 2.1 is:

[0025] Step 2.1.1: The point cloud of the ath scene in the training set is represented as Among them, p n express The nth point in n =(x n ,y n ,z n ,rf n ), (x n ,y n ,z n ) indicates p n The three-dimensional coordinates of rf n Indicates p n The reflection intensity, 1≤n≤N, N represents The number of points in x n ∈[-X w ,X w ],y n ∈[-Y w ,Y w ],z n ∈[-Z w ,Z w ], X w ,Y w ,Z w Respectively represent the maximum range of points in the point cloud on the x, y and z coordinate axes, define the voxel unit size as (dx1, dy1, dz1), and the voxel grid size as L0×W0×H0, where,

[0026] Using p n The three-dimensional coordinate x n ,y n ,z n Calculate p n The voxel index id n , id n =(ixn ,iy n ,iz n ), the calculation formula is:

[0027]

[0028] Among them, rx min ,ry min ,rz min Respectively represent the minimum value of the scene point cloud in the x, y and z directions, rx max ,ry max ,rz max They represent the maximum values ​​of the scene point cloud in the x, y and z directions respectively, and sx0, sy0 and sz0 represent the sizes of the original voxels;

[0029] Step 2.1.2: Calculate the three-dimensional coordinates and the mean of the reflection intensity of all points in the voxel to obtain the eigenvector f corresponding to the voxel. m , the original voxel dataset V after point cloud voxelization 0 ={v1,…,v m ,…,v M}, where v m Indicates V 0 The mth voxel in m ={f m ,id m}, id m Indicates v m The index number, 1≤m≤M, M represents V 0 The total number of voxels in M <N。

[0030] The specific process of step 2.3 is:

[0031] Step 2.3.1. Calculate the L-th layer sparse voxel dataset L L Medium sparse voxels The center point coordinates of V L The corresponding sparse voxel center point coordinate set Sparse Voxels The center point coordinates are expressed as The calculation formula is:

[0032]

[0033] Among them, s L represents the scale of the sparse voxels in layer L, s L =2 L-1 ;

[0034] Step 2.3.2: Sparse voxels The eigenvector of Input into the voting layer to get sparse voxels The corresponding three-dimensional coordinate offset The L-th layer sparse voxel dataset V L The offset set corresponding to the sparse voxels in is represented as offset L ,

[0035] Step 2.3.3: Sparse voxels The center point coordinates and the corresponding 3D coordinate offset Add them together to get the set of voting points Among them, ML is P vote The total number of polling stations in Represents sparse voxels The voting point is represented by The calculation formula is:

[0036]

[0037] Step 2.3.4: Calculate the voting point set P vote The corresponding voxel index number set I vote , Indicates P vote The mth voting point in The voxel index number is expressed as The calculation formula is as follows:

[0038]

[0039] Step 2.3.5, Count voting values: Define the virtual target center voxel index number set variable Ct vir , Ct vir Initialize to an empty set, count I vote The number of index numbers in the table will be greater than the set threshold τ v Index number Deposit Ct vir In, Ct vir Expressed as in, Ct vir The index number of the qth virtual target center voxel in , 1≤q≤Q, Q represents the total number of voxels in the center of the virtual target, Q <ML;

[0040] Step 2.3.6, virtual target center voxel filling: Xt vir The index number of the center voxel of the qth virtual target in Expand the voxel index number according to the neighborhood radius Δ to obtain the expanded voxel index number of the virtual target center 1≤Δ≤d Δ , d Δ Represents the maximum neighborhood radius, with radius Δ Expansion in three directions: X, Y and Z, where

[0041] The expanded virtual target center voxel index set is expressed as Where Q′ represents the total number of expanded virtual target center voxels;

[0042] Step 2.3.7, virtual target center voxel representation: the index number of the expanded virtual target center voxel The corresponding virtual target center voxel Define the eigenvector Initialized to size 1×D L The zero vector of D L ∈R, the expanded virtual target center voxel data set is expressed as:

[0043]

[0044] Step 2.3.8: The L-th layer sparse voxel dataset V L and the expanded virtual target center voxel dataset V Ex Splice to get an expanded sparse voxel dataset

[0045]

[0046] Among them, Concat represents the concatenation function;

[0047] Step 2.3.9, feature update: expand the sparse voxel dataset As input, call the open source Spconv library function SpconvConvTensor to get The corresponding tensor F L , F L As input, it is sent to the residual network layer to obtain sparse voxel enhanced features

[0048]

[0049] Among them, SubMConv3d represents the convolution function with a kernel size of 3×3×3.

[0050] The specific process of step 2.5 is:

[0051] Step 2.5.1, define the sparse sampling layer loop variable l, initialize it to 1, that is, l = 1;

[0052] Step 2.5.2: Using training set D train Midfield attractions collection Pt train Calculate the l-th layer sparse voxel dataset V l Medium sparse voxels The centroid The sparse voxel representation is revised to The calculation formula of sparse body centroid is:

[0053]

[0054] in, Represented in sparse voxels Internal points, Represented in sparse voxels Internal Points the number of

[0055] Step 2.5.3: Propose a box based on the three-dimensional proposal box set PBox Category number Sets the radius of the sphere for the spherical sampling area Proposal Box The center point of by As the center point, Sample the sparse volume centroid for the sphere radius and obtain the sampled sparse volume centroid set Where 1≤n c ≤N c , N c Indicates the total number of sampled sparse body centroids;

[0056] Step 2.5.4: Call the clustering function hdbscan.HDBSCAN() in the Python clustering algorithm library hdbscan to perform clustering on the sparse volume centroid set C l To perform K-class clustering, we get a set of point clusters: Point Cluster Expressed as Among them, N k represents the number of sparse body centroids within the kth class;

[0057] Step 2.5.5: Point clusters For each sparse volume centroid in the 3D proposal box, the point cluster is divided according to whether each sparse volume centroid is in the 3D proposal box. Divide into a set of sparse voxels within the proposal box and the sparse voxel set outside the proposal box represents the number of centroids of sparse voxels within the proposal box, represents the number of centroids of sparse voxels outside the proposal box, According to and obtain the corresponding sparse voxel features according to the index numbers, and obtain the sparse voxel feature set within the proposal box and the sparse voxel feature set outside the proposal box

[0058] Step 2.5.6: Assign the sparse voxel feature set within the proposal box to variable Q, and assign the sparse voxel feature set outside the proposal box to K and V respectively. Among them, Q, K, and V respectively represent Query, Key, and Value sent into the attention network Transformer, execute the multi-head attention function MultiHeadAttention in Pytorch, and output the features enhanced by attention Send into the feed-forward neural network FNN to obtain the enhanced features

[0059]

[0060]

[0061] Use the enhanced features to replace the feature vector to obtain class-aware enhanced sparse voxels The class-aware enhanced sparse voxel dataset of the l-th layer is

[0062] Step 2.5.7: If the variable l satisfies the condition 1 ≤ l < L, then l = l + 1, and return to Step 2.5.2; otherwise, output the class-aware enhanced sparse voxel dataset

[0063] The specific process of Step 2.6 is as follows:

[0064] Step 2.6.1: Divide the 3D proposal boxes in the 3D proposal box set PBox into grids of size 6×6×6 to obtain the grid center point set Among them, g u is the u-th grid, 1 ≤ u ≤ U, and U represents the total number of grids, represents the grid center point of the u-th grid g u ;

[0065] Step 2.6.2: Grid center point As the center, set the sphere radius r s , establish a three-dimensional sampling spherical neighborhood exist According to the sparse body mass center and the center point of the grid The distance between them is based on the principle of sampling first when the distance is closer. The class-aware enhanced sparse voxel dataset is obtained from the lth layer. Medium Sampling Sparse Body Mass and get Corresponding enhanced features The sampled sparse body mass centroid and the grid center point The distance between merge And the enhanced features of sampling Get mesh features

[0066]

[0067] Step 2.6.3: Mesh features Perform full connection and pooling operations to obtain grid pooling features:

[0068]

[0069] Among them, MLP1,…,MLP k represents k cascaded multi-layer perceptron fully connected layers, and Maxpool represents the maximum pooling layer;

[0070] Step 2.6.4: Add the proposal frame All grid pooling features are concatenated to obtain the proposed box features:

[0071]

[0072] Step 2.6.5. Finally, all proposal box features are combined into a proposal box feature set

[0073] The specific process of step 3 is:

[0074] Loss function L of outdoor 3D object detection network net It consists of voting loss, positioning loss in the proposal frame generation stage, and positioning loss in the proposal frame refinement stage. The formula is as follows:

[0075] L net =Lvote +L rpn +L rcnn (14)

[0076] Among them, L vote represents voting loss, L rpn represents the loss of the proposal box generation stage, L rcnn represents the loss of the proposal box refinement stage;

[0077] Voting loss L vote The calculation formula is as follows:

[0078]

[0079] Among them, the voxels located in the box are identified as foreground voxels, and the voxels located outside the box are identified as background voxels. V L Foreground voxels and background voxels vt fg Represents the foreground voxel The voting points generated, gt center Represents the center point of the truth box, vt bg Represents background voxels The voting points generated, vP bg Represents background voxels The voxel center point,

[0080] Positioning loss L in the proposal box generation stage rpn Divided into the proposal box classification loss L propCls and the proposal box regression loss L propReg Two parts, as follows:

[0081] L rpn =L propCls +L propReg (16)

[0082] Proposal box classification loss L propCls The calculation formula is:

[0083]

[0084] in, Represents the category score of each proposal box predicted by the proposal box generation module;

[0085] Proposal box regression loss L propReg The calculation formula is:

[0086]

[0087] in, Represents the proposal box generated by the proposal box generation module, represents the truth box;

[0088] The localization loss L in the proposal refinement stage rcnn Including IoU loss L iou and target box regression loss L bboxReg Two parts, as follows:

[0089] L rcnn =L iou +L bboxReg (19)

[0090] IoU loss L iou The calculation formula is:

[0091]

[0092] Target box regression loss L bboxReg The calculation formula is:

[0093]

[0094] in, represents the proposal box generated by the target refinement module, Represents a truth box.

[0095] The specific process of step 4 is:

[0096] Step 4.1, set the training parameters of the outdoor 3D target detection network structure Net, including the total number of iterations total_epoch, the learning rate lr, the number of data samples used in each batch training batch_size, the minimum change value of the target loss function ΔMini_Loss, the training iteration number variable epoch and the network optimizer Adam;

[0097] Step 4.2: From the training dataset D train Randomly read batch_size samples and send them to the outdoor 3D target detection network structure Net for training;

[0098] Step 4.3: Calculate the loss function L for two iterations during training net The absolute difference diff, if diff<ΔMini_Loss or epoch>total_epoch, then the training ends and the network model Model is output train ; Otherwise, epoch = epoch + 1, use the Adam optimizer to reversely correct the weight coefficients of each network layer in the outdoor three-dimensional target detection network structure Net, and repeat steps 4.2-4.3.

[0099] The specific process of step 5 is:

[0100] Step 5.1: Input test set D test = {Pt test ,Gt test}, define the total number of scenes in the test set variable B, scene count variable b, and three-dimensional object detection result set Pred test , initialize b = 1, Pred test is an empty set;

[0101] Step 5.2: From D test Extract the bth scene point cloud Input network model Model train , get the three-dimensional target detection result set pred after network processing b , pred b = {BBox b ,Conf b};

[0102] Step 5.3: Set the detection results to pred b Append to Pred test In Pred test =Pred test +pred b ;

[0103] Step 5.4: Determine whether b is less than B. If so, b = b + 1 and return to step 5.2; otherwise, output Pred test ={pred1,…,pred b ,…,pred B};

[0104] The specific process of step 6 is:

[0105] The average mean precision mAP is used as the main evaluation indicator of network performance, and the evaluation result pa is obtained. The calculation formula is:

[0106] pa=mAP(Pred test ,Gt test ) (twenty two)

[0107] Among them, mAP(*,*) is the calculation function of the average precision of yolo open source, Pred test and Gt test They respectively represent the three-dimensional target detection result set and the corresponding true value label test set.

[0108] The beneficial effects of the present invention are:

[0109] 1. The present invention is based on the outdoor 3D target detection method based on target center and class perception. The target center point cloud generation module and the target class perception feature enhancement module are embedded into the two-stage outdoor 3D target detection network framework, which can effectively improve the 3D target detection accuracy.

[0110] 2. Based on the target center and class perception outdoor 3D target detection method, the present invention proposes a target class perception feature enhancement method. This method adaptively obtains local point clouds of targets of different sizes according to different target types, clusters the local point clouds by position, and enhances the feature distinction of point clouds within clusters based on the attention mechanism, thereby enhancing the network's perception of targets.

[0111] 3. The present invention proposes a target center-aware feature enhancement method based on the target center and class-aware outdoor three-dimensional target detection method. The method generates virtual target center voxels by using the voting mechanism of the target surrounding voxels for the target center area, and diffuses the feature information around or on the surface of the target to the target center by using sparse convolution. In addition, since there are a large number of background voxels in the scene and only a small number of foreground voxels can determine the target center, the present invention designs voting losses for the foreground and background voxels respectively to guide the network to generate foreground voxels with votes as close to the target center as possible, thereby reducing the generated target center noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Figure 1 It is a flow chart of an outdoor three-dimensional target detection method based on target center and class perception of the present invention;

[0113] Figure 2 It is a network structure diagram of the outdoor three-dimensional target detection method based on target center and class perception of the present invention;

[0114] Figure 3 is a true value result image of a target in a point cloud scene extracted from the test set of Example 1;

[0115] Figure 4 The existing Voxel-RCNN method is Figure 3 Target detection result diagram in the same point cloud scene;

[0116] Figure 5 The prior art PDV method is Figure 3 Target detection result diagram in the same point cloud scene;

[0117] Figure 6 The outdoor three-dimensional target detection method based on target center and class perception of the present invention is Figure 3 Target detection results in the same point cloud scene. DETAILED DESCRIPTION

[0118] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0119] Example 1

[0120] The present invention is based on the outdoor three-dimensional target detection method of target center and class perception, the process is as follows Figure 1 As shown, the specific implementation steps are as follows:

[0121] Step 1: Get the KITTI scene dataset D and divide it into training set D train and the test set D test , specifically:

[0122] Download the KITTI dataset D from the public dataset website, D = {Pt, Gt}, where Pt represents the scene point cloud set and Gt represents the true value dataset. Divide the dataset D into training set D according to the ratio of A:B. train = {Pt train ,Gt train} and test set D test = {Pt test ,Gt test},in,

[0123] represents the point cloud of the ath scene in the training set, represents the target truth value of the ath scene in the training set,

[0124]

[0125] represents the point cloud of the bth scene in the test set, represents the target true value of the b-th scene in the test set, 1≤a≤A, 1≤b≤B, A is the total number of scenes in the training set, B is the total number of scenes in the test set, and the values ​​of A and B are 3712 and 3769 respectively;

[0126] Step 2: Construct an outdoor 3D target detection network structure Net based on target center and class perception. The network structure is as follows: Figure 2 As shown, specifically:

[0127] Step 2.1: Construct a point cloud voxelization module to convert the point cloud of the ath scene in the training set into Perform point cloud voxelization to obtain the original voxel dataset V 0 , specifically:

[0128] Step 2.1.1: The point cloud of the ath scene in the training set is represented as Among them, p n express The nth point in n =(x n ,y n ,z n ,rf n ), (x n ,y n ,z n ) indicates p n The three-dimensional coordinates of rf n Indicates p n The reflection intensity, 1≤n≤N, N represents The number of points in x n ∈[-X w ,X w ],y n ∈[-Y w ,Y w ],z n ∈[-Z w ,Z w ], X w ,Y w ,Z w Respectively represent the maximum range of points on the x, y and z coordinate axes of the point cloud, with specific values ​​of 70, 40 and 3. The voxel unit size is defined as (dx1, dy1, dz1), and the voxel grid size is L0×W0×H0, where

[0129]

[0130] Using p n The three-dimensional coordinate x n ,y n ,z n Calculate p n The voxel index id n , id n =(ix n ,iy n ,iz n ), the calculation formula is:

[0131]

[0132] Among them, rx min ,ry min ,rz min They represent the minimum values ​​of the scene point cloud in the x, y, and z directions, with specific values ​​of 0, -40, and -3, respectively. max ,ry max ,rz maxThey represent the maximum values ​​of the scene point cloud in the x, y and z directions, with specific values ​​of 70, 40 and 1, respectively. sx0, sy0 and sz0 represent the sizes of the original voxels, with specific values ​​of 1600, 1408 and 41;

[0133] Step 2.1.2: Calculate the three-dimensional coordinates and the mean of the reflection intensity of all points in the voxel to obtain the eigenvector f corresponding to the voxel. m , the original voxel dataset V after point cloud voxelization 0 ={v1,…,v m ,…,v M}, where v m Indicates V 0 The mth voxel in m ={f m ,id m}, id m Indicates v m The index number, 1≤m≤M, M represents V 0 The total number of voxels in M <N;

[0134] Step 2.2: Construct a voxel feature extraction module and convert the original voxel dataset V 0 Input into the sparse 3D CNN network for multi-scale downsampling voxel feature extraction. The sparse 3D CNN network structure is shown in Table 1. The sparse voxel dataset V with different downsampling scales of L layers is obtained. Net = {V 1 ,…,V l ,…,V L},in, V l is the sparse voxel dataset of the lth layer, 1≤l≤L, L is the total number of downsampling layers, L=4, Ml represents the number of non-empty voxels in the lth layer, and sparse voxels Expressed as Represents sparse voxels The feature vector of l , D l ∈R, Represents sparse voxels The index number in the x, y and z directions is expressed as

[0135] Table 1

[0136]

[0137] Step 2.3: Build the target center point cloud generation module and transform the L-th layer sparse voxel dataset V L Perform voting and filling to obtain an expanded sparse voxel dataset and the corresponding sparse voxel enhanced features in, V Ex Represents the expanded virtual center voxel dataset, specifically:

[0138] Step 2.3.1. Calculate the L-th layer sparse voxel dataset V L Medium sparse voxels The center point coordinates of V L The corresponding sparse voxel center point coordinate set Sparse Voxels The center point coordinates are expressed as The calculation formula is:

[0139]

[0140] Among them, s L represents the scale of the sparse voxels in layer L, s L =2 L-1 =8;

[0141] Step 2.3.2: Sparse voxels The eigenvector of Input into the voting layer to get sparse voxels The corresponding three-dimensional coordinate offset The L-th layer sparse voxel dataset V L The offset set corresponding to the sparse voxels in is represented as offset L , The network structure of the voting layer is shown in Table 2:

[0142] Table 2

[0143]

[0144] Step 2.3.3: Sparse voxels The center point coordinates and the corresponding 3D coordinate offset Add them together to get the set of voting points Among them, ML is P vote The total number of polling stations in Represents sparse voxels The voting point is represented by The calculation formula is:

[0145]

[0146] Step 2.3.4: Calculate the voting point set P vote The corresponding voxel index number set I vote , Indicates P vote The mth voting point in The voxel index number is expressed as The calculation formula is as follows:

[0147]

[0148] Step 2.3.5, Count voting values: Define the virtual target center voxel index number set variable Ct vir , Ct vir Initialize to an empty set, count I vote The number of index numbers in the table will be greater than the set threshold τ v Index number Deposit Ct vir In, Ct vir Expressed as in, Ct vir The index number of the qth virtual target center voxel in, 1≤q≤Q, Q represents the total number of voxels in the center of the virtual target, Q <ML,τ v =5;

[0149] Step 2.3.6, virtual target center voxel filling: Ct vir The index number of the center voxel of the qth virtual target in Expand the voxel index number according to the neighborhood radius Δ to obtain the expanded voxel index number of the virtual target center 1≤Δ≤d Δ , d Δ represents the maximum neighborhood radius, d Δ =2, with radius Δ Expansion in three directions: X, Y and Z, where

[0150] The expanded virtual target center voxel index set is expressed as Where Q′ represents the total number of expanded virtual target center voxels; Step 2.3.7, virtual target center voxel representation: is the index number of the expanded virtual target center voxel The corresponding virtual target center voxel Define the eigenvector Initialized to size 1×D L The zero vector of D L ∈R, the expanded virtual target center voxel data set is expressed as:

[0151]

[0152] Step 2.3.8: The L-th layer sparse voxel dataset V L and the expanded virtual target center voxel dataset V Ex Splice to get an expanded sparse voxel dataset

[0153]

[0154] Among them, Concat represents the concatenation function;

[0155] Step 2.3.9, feature update: expand the sparse voxel dataset As input, call the open source Spconv library function SpconvConvTensor to get The corresponding tensor F L , F L As input, it is sent to the residual network layer to obtain sparse voxel enhanced features

[0156]

[0157] Among them, SubMConv3d represents the convolution function with a kernel size of 3×3×3;

[0158] Step 2.4: Construct a proposal box generation module to enhance the sparse voxel features As input, the function view() in the Pytorch library is used to enhance the features of sparse voxels. Convert to bird's eye view feature map Bird's-eye view feature map Input into the region proposal network under the open source Openpcdet library to obtain a set of 3D proposal boxes Where 1≤n p ≤N p , N p Represents the total number of 3D proposal boxes,

[0159]

[0160] Indicates a proposal box The center coordinates of The proposal boxes The height, width and length of Indicates a proposal box The z-axis rotation angle, Indicates a proposal box The category number, Indicates a proposal box Category score of

[0161] Step 2.5: Construct a target class perception feature enhancement module to perform Net = {V 1 ,…,V l ,…,V L}Sampling and clustering are performed to obtain a class-aware enhanced sparse voxel dataset Specifically:

[0162] Step 2.5.1, define the sparse sampling layer loop variable l, initialize it to 1, that is, l = 1;

[0163] Step 2.5.2: Using training set D tiain Midfield attractions collection Pt train Calculate the l-th layer sparse voxel dataset V l Medium sparse voxels The centroid The sparse voxel representation is revised to The calculation formula of sparse body centroid is:

[0164]

[0165] in, Represented in sparse voxels Internal points, Represented in sparse voxels Internal Points the number of

[0166] Step 2.5.3: Propose a box based on the three-dimensional proposal box set PBox Category number Sets the radius of the sphere for the spherical sampling area Proposal Box The center point of by As the center point, Sample the sparse volume centroid for the sphere radius and obtain the sampled sparse volume centroid set Where 1≤n c ≤N c , N c Indicates the total number of sampled sparse body centroids;

[0167] Step 2.5.4: Call the clustering function hdbscan.HDBSCAN() in the Python clustering algorithm library hdbscan to perform clustering on the sparse volume centroid set C l To perform K-class clustering, we get a set of point clusters: Point Cluster Expressed as Among them, Nk Denote the number of the centroids of the sparse voxels within the k-th class;

[0168] Step 2.5.5, for each centroid of the sparse voxels in the point cluster According to whether each centroid of the sparse voxels is within the three-dimensional proposal box, divide the point cluster into the set of sparse voxels within the proposal box and the set of sparse voxels outside the proposal box Denote the number of the centroids of the sparse voxels within the proposal box, Denote the number of the centroids of the sparse voxels outside the proposal box, According to and to obtain the corresponding sparse voxel features according to the index numbers, and get the set of sparse voxel features within the proposal box and the set of sparse voxel features outside the proposal box

[0169] Step 2.5.6, assign the set of sparse voxel features within the proposal box to the variable Q, and assign the set of sparse voxel features outside the proposal box to K and V respectively, where Q, K, and V respectively represent Query, Key, and Value fed into the attention network Transformer, execute the multi-head attention function MultiHeadAttention in Pytorch, and output the features enhanced by attention Feed into the feed-forward neural network FNN to obtain the enhanced features

[0170]

[0171]

[0172] Use the enhanced features to replace the feature vector to obtain the class-aware enhanced sparse voxels The dataset of the class-aware enhanced sparse voxels at the l-th layer is

[0173] Step 2.5.7, if the variable l satisfies the condition 1 ≤ l < L, then l = l + 1, and return to Step 2.5.2; otherwise, output the dataset of the class-aware enhanced sparse voxels

[0174] Step 2.6: Construct the target area grid pooling module to enhance the class-aware sparse voxel dataset Gridding and sampling with the three-dimensional proposal box set PBox to obtain the proposal box feature set F PBox , specifically:

[0175] Step 2.6.1: Add the three-dimensional proposal box in the three-dimensional proposal box set PBox Divide into a 6×6×6 grid and get the grid center point set Among them, g u is the u-th grid, 1≤u≤U, U represents the total number of grids, Represents the uth grid g u The center point of the grid;

[0176] Step 2.6.2: Grid center point As the center, set the sphere radius r s =4.0, establish a three-dimensional sampling spherical neighborhood exist According to the sparse body mass center and the grid center point The distance between them is based on the principle of sampling first when the distance is closer. The class-aware enhanced sparse voxel dataset is obtained from the lth layer. Medium Sampling Sparse Body Mass and get Corresponding enhanced features The sampled sparse body mass centroid and the grid center point The distance between merge And the enhanced features of sampling Get mesh features

[0177]

[0178] Step 2.6.3: Mesh features Perform full connection and pooling operations to obtain grid pooling features:

[0179]

[0180] Among them, MLP1,…,MLP k represents k cascaded multi-layer perceptron fully connected layers, and Maxpool represents the maximum pooling layer;

[0181] Step 2.6.4: Add the proposal frame All grid pooling features are concatenated to obtain the proposed box features:

[0182]

[0183] Step 2.6.5. Finally, all proposal box features are combined into a proposal box feature set

[0184] Step 2.7: Construct the target refinement module and transform the proposed box feature set F pBox Perform target refinement prediction to obtain the refined proposal box BBox and confidence Conf, where:

[0185] Represents the category score of the proposal box after target refinement, 1≤n b ≤N b , N b Represents the total number of proposal boxes after target refinement, N b =128, Represents the center coordinates of the proposal box after target refinement, Respectively represent the height, width, and length of the proposed box after target refinement, Indicates the z-axis rotation angle of the proposal box;

[0186] The target refinement module contains a regression network branch and a confidence prediction network branch, as shown in Tables 3 and 4 below:

[0187] Table 3

[0188]

[0189] Table 4

[0190]

[0191]

[0192] Step 3: Construct the loss function, specifically:

[0193] Loss function L of outdoor 3D object detection network net It consists of voting loss, positioning loss in the proposal frame generation stage, and positioning loss in the proposal frame refinement stage. The formula is as follows:

[0194] L net =L vote +L rpn +L rcnn (14)

[0195] Among them, L vote represents voting loss, L rpn represents the loss of the proposal box generation stage, L rcnnrepresents the loss of the proposal box refinement stage;

[0196] Voting loss L vote The calculation formula is as follows:

[0197]

[0198] Among them, the voxels located in the box are identified as foreground voxels, and the voxels located outside the box are identified as background voxels. V L Foreground voxels and background voxels vt fg Represents the foreground voxel The voting points generated, gt center Represents the center point of the truth box, vt bg Represents background voxels The voting points generated, vP bg Represents background voxels The voxel center point, vt fg ,vt bg ∈P vote ,vP bg ∈P ctr ;

[0199] Positioning loss L in the proposal box generation stage rpn Divided into the proposal box classification loss L propCls and the proposal box regression loss L propReg Two parts, as follows:

[0200] L rpn =L propcls +L propReg (16)

[0201] Proposal box classification loss L propCls The calculation formula is:

[0202]

[0203] in, Represents the category score of each proposal box predicted by the proposal box generation module;

[0204] Proposal box regression loss L propReg The calculation formula is:

[0205]

[0206] in, Represents the proposal box generated by the proposal box generation module, represents the truth box;

[0207] The localization loss L in the proposal refinement stagercnn Including IoU loss L iou and target box regression loss L bboxReg Two parts, as follows:

[0208] L rcnn =L iou +L bboxReg (19)

[0209] IoU loss L iou The calculation formula is:

[0210]

[0211] Target box regression loss L bboxReg The calculation formula is:

[0212]

[0213] in, represents the proposal box generated by the target refinement module, represents the truth box;

[0214] Step 4: Set the training parameters and set the training set D train Input the outdoor 3D target detection network structure Net for training and obtain the network model Model train , specifically:

[0215] Step 4.1, set the training parameters of the outdoor 3D target detection network structure Net, the training parameters include total iterations total_epoch = 80, learning rate lr = 0.01, the number of data samples used in each batch training batch_size = 4, the minimum change value of the target loss function ΔMini_Loss, the training iteration number variable epoch and the network optimizer Adam;

[0216] Step 4.2: From the training dataset D train Randomly read batch_size samples and send them to the outdoor 3D target detection network structure Net for training;

[0217] Step 4.3: Calculate the loss function L for two iterations during training net The absolute difference diff, if diff<ΔMini_Loss or epoch>total_epoch, then the training ends and the network model Model is output train ; Otherwise, epoch = epoch + 1, use the Adam optimizer to reversely correct the weight coefficients of each network layer in the outdoor three-dimensional target detection network structure Net, and repeat steps 4.2-4.3.

[0218] Step 5: Use the network model Model to detect the outdoor three-dimensional target network structure Net train In the test set D test Perform target detection on it and output the detection result Pred test , specifically:

[0219] Step 5.1: Input test set D test = {Pt test ,Gt test}, define the total number of scenes in the test set variable B, scene count variable b, and three-dimensional object detection result set Pred test , initialize b = 1, Pred test is an empty set;

[0220] Step 5.2: From D test Extract the bth scene point cloud Input network model Model train , get the three-dimensional target detection result set pred after network processing b , pred b = {BBox b ,Conf b};

[0221] Step 5.3: Set the detection results to pred b Append to Pred test In Pred test =Pred test +pred b ;

[0222] Step 5.4: Determine whether b is less than B. If so, b = b + 1 and return to step 5.2; otherwise, output Pred test ={pred1,…,pred b ,…,pred B};

[0223] Step 6: According to the detection result Pred test Network Model train The detection performance of

[0224] The average mean precision mAP is used as the main evaluation indicator of network performance, and the evaluation result pa is obtained. The calculation formula is:

[0225] pa=mAP(Pred test ,Gt test ) (twenty two)

[0226] Among them, mAP(*,*) is the calculation function of the average precision of yolo open source, Predtest and Gt test They respectively represent the three-dimensional target detection result set and the corresponding true value label test set.

[0227] In this embodiment, a three-dimensional object detection effect test is performed on the public 3D vision dataset Kitti, and the test results are evaluated subjectively and objectively.

[0228] The KITTI dataset is a widely used dataset for autonomous driving and computer vision research. It contains a large amount of data of various types collected in real-world scenes, including stereo images, lidar point clouds, visual odometers, stereo image sequences, optical flows, vehicle and pedestrian annotations, etc. The KITTI dataset provides a training set and a test set, which contain 7481 and 7518 sets of sample data respectively. The data are annotated with 8 detection categories. Most of the existing outdoor 3D target detection methods mainly use three types of targets: cars, pedestrians, and cyclists to evaluate algorithm performance. For each type of target, the KITTI dataset divides the target into three different difficulty levels: simple, medium, and difficult according to the degree of occlusion and truncation ratio of the target in the point cloud scene.

[0229] Subjective evaluation:

[0230] The 3D detection results of the targets in the scenes in the test set are as follows: Figure 4-Figure 6 As shown, Figure 3 represents the true value annotation of a target in a point cloud scene in the test set, Figure 4 Represents the detection result of the Voxel-RCNN algorithm. The white box is the target detection box. Figure 5 is the detection result of the PDV algorithm, the white box is the target detection box, Figure 6 represents the detection result of the present invention, and the white one is the target detection frame. Figure 4-Figure 6 The test results and Figure 3 It can be seen from the comparison of the true values ​​that the target center point cloud generation module and the target class perception module proposed in the present invention play a positive role in improving the target detection accuracy of the network and can obtain better three-dimensional target detection results.

[0231] Objective evaluation:

[0232] The common detection evaluation indicators on the KITTI dataset include: 3D average precision (Mean AveragePrecision, mAP 3D ), bird's-eye view average precision (mAP bev), etc., the present invention mainly detects three categories of targets, namely "car", "pedestrian" and "cyclist" in the Kitti dataset, uses the 3D average precision as the evaluation indicator, sets the IoU thresholds of the three categories of samples of "car", "pedestrian" and "cyclist" to {0.7, 0.5, 0.5}, evaluates the overlap between the network output and the true value, and determines whether the network prediction is correct.

[0233] Table 4

[0234]

[0235] According to the data in Table 4, compared with the classic Baseline in the past, the algorithm in this paper has improved the detection difficulty of Moderate and Hard in the Car category by 0.75% and 0.50% respectively, the detection difficulty of Easy, Moderate and Hard in the Pedestrian category by 1.60%, 1.24% and 1.31% respectively, and the detection difficulty of Easy, Moderate and Hard in the Cyclist category by 0.96%, 1.08% and 0.77% respectively; compared with the more classic BtcDet algorithm in the past two years, the algorithm in this paper has improved the detection difficulty of Easy, Moderate and Hard in the Pedestrian category by 0.52%, 1.79% and 1.50% respectively, and the detection difficulty of Easy, Moderate and Hard in the Cyclist category by 1.14%, 0.49% and 0.56% respectively. This proves the effectiveness of the improved algorithm of the present invention.

Claims

1. Outdoor 3D target detection method based on target center and class perception, It is characterized in that Follow the steps below to implement it: Step 1: Obtain a 3D point cloud dataset D and divide it into a training set D train and the test set D test ; Step 2: Construct an outdoor 3D target detection network structure Net based on target center and class perception; Step 3: Define the loss function; Step 4: Set the training parameters and set the training set D train Input the outdoor 3D target detection network structure Net for training and obtain the network model Model train ; Step 5: Use the network model Model to detect the outdoor three-dimensional target network structure Net train In the test set D test Perform target detection on it and output the detection result Pred test ; Step 6: According to the detection result Pred test Network Model train The detection performance was evaluated.

2. The outdoor three-dimensional target detection method based on target center and class perception according to claim 1, It is characterized in that The specific process of step 1 is as follows: Download the 3D point cloud dataset D from the official website of the public dataset, D = {Pt, Gt}, where Pt represents the scene point cloud set and Gt represents the true value dataset. Divide the dataset D into the training set D according to the ratio of A:B train = {Pt train ,Gt train } and test set D test = {Pt test ,Gt test },in, represents the point cloud of the ath scene in the training set, represents the target truth value of the ath scene in the training set, represents the point cloud of the bth scene in the test set, represents the target true value of the b-th scene in the test set, 1≤a≤A, 1≤b≤B, A is the total number of scenes in the training set, and B is the total number of scenes in the test set.

3. The outdoor three-dimensional target detection method based on target center and class perception according to claim 2, It is characterized in that The specific process of step 2 is: Step 2.1: Construct a point cloud voxelization module to convert the point cloud of the ath scene in the training set into Perform point cloud voxelization to obtain the original voxel dataset V 0 ; Step 2.2: Construct a voxel feature extraction module and convert the original voxel dataset V 0 Input into the sparse 3D CNN network for multi-scale downsampling voxel feature extraction, and obtain L layers of sparse voxel datasets V with different downsampling scales. Net = {V 1 ,…,V l ,…,V L },in, V l is the sparse voxel dataset of the lth layer, 1≤l≤L, L is the total number of downsampling layers, Ml represents the number of non-empty voxels in the lth layer, and sparse voxels Expressed as Represents sparse voxels The feature vector of l , D l ∈T, Represents sparse voxels The index number in the x, y and z directions is expressed as Step 2.3: Build the target center point cloud generation module and transform the L-th layer sparse voxel dataset V L Voting and expansion are performed to obtain an expanded sparse voxel dataset and the corresponding sparse voxel enhanced features in, V Ex represents the augmented virtual target center voxel dataset; Step 2.4: Construct a proposal box generation module to enhance the sparse voxel features As input, the function view() in the Pytorch library is used to enhance the features of sparse voxels. Convert to bird's eye view feature map Bird's-eye view feature map Input into the region proposal network under the open source Openpcdet library to obtain a set of 3D proposal boxes Where 1≤n p ≤N p , N p Represents the total number of 3D proposal boxes, Indicates a proposal box The center coordinates of Represents the proposal frame The height, width and length of Indicates a proposal box The z-axis rotation angle, Indicates a proposal box The category number, Indicates a proposal box Category score of Step 2.5: Construct a target class perception feature enhancement module to perform Net = {V 1 ,…,V l ,…,V L }Sampling and clustering are performed to obtain a class-aware enhanced sparse voxel dataset Step 2.6: Construct the target area grid pooling module to enhance the class-aware sparse voxel dataset Gridding and sampling with the three-dimensional proposal box set PBox to obtain the proposal box feature set F PBox ; Step 2.7: Construct the target refinement module and transform the proposed box feature set F pBox Perform target refinement prediction to obtain the refined proposal box BBox and confidence Conf, where: Represents the category score of the proposal box after target refinement, 1≤n b ≤N b , N b Represents the total number of proposal boxes after target refinement, Represents the center coordinates of the proposal box after target refinement, Respectively represent the height, width, and length of the proposed box after target refinement, Indicates the z-axis rotation angle of the proposal box.

4. The outdoor three-dimensional target detection method based on target center and class perception according to claim 3, It is characterized in that The specific process of step 2.1 is as follows: Step 2.1.1: The point cloud of the ath scene in the training set is represented as Among them, p n express The nth point in n =(x n ,y n ,z n ,rf n ), (x n ,y n ,z n ) indicates p n The three-dimensional coordinates of rf n Indicates p n The reflection intensity, 1≤n≤N, N represents The number of points in x n ∈[-X w ,X w ],y n ∈[-Y w ,Y w ],z n ∈[-Z w ,Z w ], X w ,Y w ,Z w Respectively represent the maximum range of points in the point cloud on the x, y and z coordinate axes, and define the voxel unit size as (dx 1 ,dy 1 ,dz 1 ), the voxel grid size is L 0 ×W 0 ×H 0 ,in, Using p n The three-dimensional coordinate x n ,y n ,z n Calculate p n The voxel index id n , id n =(ix n ,iy n ,iz n ), the calculation formula is: Among them, rx min ,ry min ,rz min Respectively represent the minimum value of the scene point cloud in the x, y and z directions, rx max ,ry max ,rz max Respectively represent the maximum value of the scene point cloud in the x, y and z directions, sx 0 ,sy 0 ,sz 0 represents the size of the original voxel; Step 2.1.2: Calculate the three-dimensional coordinates and the mean of the reflection intensity of all points in the voxel to obtain the eigenvector f corresponding to the voxel. m , the original voxel dataset V after point cloud voxelization 0 = {v 1 ,…,v m ,…,v M }, where v m Indicates V 0 The mth voxel in m ={f m ,id m }, id m Indicates v m The index number, 1≤m≤M, M represents V 0 The total number of voxels in M <N。 5. The outdoor three-dimensional target detection method based on target center and class perception according to claim 4, It is characterized in that The specific process of step 2.3 is as follows: Step 2.3.

1. Calculate the L-th layer sparse voxel dataset V L Medium sparse voxels The center point coordinates of V L The corresponding sparse voxel center point coordinate set Sparse Voxels The center point coordinates are expressed as The calculation formula is: Among them, s L represents the scale of the sparse voxels in layer L, s L =2 L-1 ; Step 2.3.2: Sparse voxels The eigenvector of Input into the voting layer to get sparse voxels The corresponding three-dimensional coordinate offset The L-th layer sparse voxel dataset V L The offset set corresponding to the sparse voxels in is represented as offset L , Step 2.3.3: Sparse voxels The center point coordinates and the corresponding 3D coordinate offset Add them together to get the set of voting points Among them, ML is P vote The total number of polling stations in Represents sparse voxels The voting point is represented by The calculation formula is: Step 2.3.4: Calculate the voting point set P vote The corresponding voxel index number set I vote , Indicates P vote The mth voting point in The voxel index number is expressed as The calculation formula is as follows: Step 2.3.5, Count voting values: Define the virtual target center voxel index number set variable Ct vir , Ct vir Initialize to an empty set, count I vote The number of index numbers in the table will be greater than the set threshold τ v Index number Deposit Ct vir In, Ct vir Expressed as in, Ct vir The index number of the qth virtual target center voxel in , Q represents the total number of voxels in the center of the virtual target, Step 2.3.6, virtual target center voxel filling: Ct vir The index number of the center voxel of the qth virtual target in Expand the voxel index number according to the neighborhood radius Δ to obtain the expanded voxel index number of the virtual target center d Δ Represents the maximum neighborhood radius, with radius Δ Expansion in three directions: X, Y and Z, where The expanded virtual target center voxel index set is expressed as Where Q′ represents the total number of expanded virtual target center voxels; Step 2.3.7, virtual target center voxel representation: the index number of the expanded virtual target center voxel The corresponding virtual target center voxel Define the eigenvector Initialized to size 1×D L The zero vector of D L ∈R, the expanded virtual target center voxel data set is expressed as: Step 2.3.8: The L-th layer sparse voxel dataset V L and the expanded virtual target center voxel dataset V Ex Splice to get an expanded sparse voxel dataset Among them, Concat represents the concatenation function; Step 2.3.9, feature update: expand the sparse voxel dataset As input, call the open source Spconv library function SpconvConvTensor to get The corresponding tensor F L , F L As input, it is sent to the residual network layer to obtain sparse voxel enhanced features Among them, SubMConv3d represents the convolution function with a kernel size of 3×3×3.

6. The outdoor three-dimensional target detection method based on target center and class perception according to claim 5, It is characterized in that The specific process of step 2.5 is as follows: Step 2.5.1, define the sparse sampling layer loop variable l, initialize it to 1, that is, l = 1; Step 2.5.2: Using training set D train Midfield attractions collection Pt train Calculate the l-th layer sparse voxel dataset V l Medium sparse voxels The centroid The sparse voxel representation is revised to The calculation formula of sparse body centroid is: in, Represented in sparse voxels Internal points, Represented in sparse voxels Internal Points the number of Step 2.5.3: Propose a box based on the three-dimensional proposal box set PBox Category number Sets the radius of the sphere for the spherical sampling area Proposal Box The center point of by As the center point, Sample the sparse volume centroid for the sphere radius and obtain the sampled sparse volume centroid set Where 1≤n c ≤N c , N c Indicates the total number of sampled sparse body centroids; Step 2.5.4: Call the clustering function hdbscan.HDBSCAN() in the Python clustering algorithm library hdbscan to perform clustering on the sparse volume centroid set C l To perform K-class clustering, we get a set of point clusters: Point Cluster Expressed as Among them, N k represents the number of sparse body centroids within the kth class; Step 2.5.5: Point clusters For each sparse volume centroid in the 3D proposal box, the point cluster is divided according to whether each sparse volume centroid is in the 3D proposal box. Divide into a set of sparse voxels within the proposal box and the set of sparse voxels outside the proposal box represents the number of sparse body centroids within the proposal box, represents the number of sparse body centroids outside the proposal box, according to and Get the corresponding sparse voxel feature by the index number, and get the sparse voxel feature set in the proposal box And the sparse voxel feature set outside the proposal box Step 2.5.6: Set the sparse voxel features in the proposal box Assign to variable Q, and propose a sparse voxel feature set outside the box Assign values ​​to K and V respectively, where Q, K, and V represent the query, key, and value sent to the attention network Transformer, respectively, and execute the multi-head attention function MultiHeadAttention in Pytorch to output Features after attention enhancement Will Input the feedforward neural network FNN to get enhanced features Using Enhanced Features Replace the feature vector Get class-aware enhanced sparse voxels The l-th layer class-aware enhanced sparse voxel dataset is Step 2.5.7: If the variable l satisfies the condition 1 ≤ l < L, then l = l + 1, and return to Step 2.5.2; otherwise, output the class-aware enhanced sparse voxel dataset 7. The outdoor three-dimensional target detection method based on target center and class perception according to claim 6, It is characterized in that The specific process of step 2.6 is as follows: Step 2.6.1: Add the three-dimensional proposal box in the three-dimensional proposal box set PBox Divide into a 6×6×6 grid and get the grid center point set Among them, g u is the u-th grid, 1≤u≤U, U represents the total number of grids, Represents the uth grid g u The center point of the grid; Step 2.6.2: Grid center point As the center, set the sphere radius r s , establish a three-dimensional sampling spherical neighborhood exist According to the sparse body mass center and the grid center point The distance between them is based on the principle of sampling first when the distance is closer. The class-aware enhanced sparse voxel dataset is obtained from the lth layer. Medium Sampling Sparse Body Mass and get Corresponding enhanced features The sampled sparse body mass centroid and the grid center point The distance between merge And the enhanced features of sampling Get mesh features Step 2.6.3: Mesh features Perform full connection and pooling operations to obtain grid pooling features: Among them, MLP 1 ,…,MLP k represents k cascaded multi-layer perceptron fully connected layers, and Maxpool represents the maximum pooling layer; Step 2.6.4: Add the proposal frame All grid pooling features are concatenated to obtain the proposed box features: Step 2.6.

5. Finally, all proposal box features are combined into a proposal box feature set 8. The outdoor three-dimensional target detection method based on target center and class perception according to claim 7, It is characterized in that The specific process of step 3 is as follows: Loss function L of outdoor 3D object detection network net It consists of voting loss, positioning loss in the proposal frame generation stage, and positioning loss in the proposal frame refinement stage. The formula is as follows: L net =L vote +L rpn +L rcnn (14) Among them, L vote represents voting loss, L rpn represents the loss of the proposal box generation stage, L rcnn represents the loss of the proposal box refinement stage; Voting loss L vote The calculation formula is as follows: Among them, the voxels located in the box are identified as foreground voxels, and the voxels located outside the box are identified as background voxels. V L Foreground voxels and background voxels vt fg Represents the foreground voxel The voting points generated, gt center Represents the center point of the truth box, vt bg Represents background voxels The voting points generated, vP bg Represents background voxels The voxel center point, vt fg ,vt bg ∈P vote ,vP bg ∈P ctr ; Positioning loss L in the proposal box generation stage rpn Divided into the proposal box classification loss L propCls and the proposal box regression loss L propReg Two parts, as follows: L rpn =L propCls +L propReg (16) Proposal box classification loss L propCls The calculation formula is: in, Represents the category score of each proposal box predicted by the proposal box generation module; Proposal box regression loss L propReg The calculation formula is: in, Represents the proposal box generated by the proposal box generation module, represents the truth box; The localization loss L in the proposal refinement stage rcnn Including IoU loss L iou and target box regression loss L bboxReg Two parts, as follows: L rcnn =L iou +L bboxReg (19) IoU loss L iou The calculation formula is: Target box regression loss L bboxReg The calculation formula is: in, represents the proposal box generated by the target refinement module, Represents a truth box.

9. The outdoor three-dimensional target detection method based on target center and class perception according to claim 8, It is characterized in that The specific process of step 4 is as follows: Step 4.1, set the training parameters of the outdoor 3D target detection network structure Net, including the total number of iterations total_epoch, the learning rate lr, the number of data samples used in each batch training batch_size, the minimum change value of the target loss function ΔMini_Loss, the training iteration number variable epoch and the network optimizer Adam; Step 4.2: From the training dataset D train Randomly read batch_size samples and send them to the outdoor 3D target detection network structure Net for training; Step 4.3: Calculate the loss function L for two iterations during training net The absolute difference diff, if diff<ΔMini_Loss or epoch>total_epoch, then the training ends and the network model Model is output train ; Otherwise, epoch = epoch + 1, use the Adam optimizer to reversely correct the weight coefficients of each network layer in the outdoor three-dimensional target detection network structure Net, and repeat steps 4.2-4.

3.

10. The outdoor three-dimensional target detection method based on target center and class perception according to claim 9, It is characterized in that The specific process of step 5 is as follows: Step 5.1: Input test set D test = {Pt test ,Gt test }, define the total number of scenes in the test set variable B, scene count variable b, and three-dimensional object detection result set Pred test , initialize b = 1, Pred test is an empty set; Step 5.2: From D test Extract the bth scene point cloud Input network model Model train , get the three-dimensional target detection result set pred after network processing b , pred b = {BBox b ,Conf b }; Step 5.3: Set the detection results to pred b Append to Pred test In Pred test =Pred test +pred b ; Step 5.4: Determine whether b is less than B. If so, b = b + 1 and return to step 5.2; otherwise, output Pred test ={pred 1 ,…,pred b ,…,pred B }; The specific process of step 6 is as follows: The average mean precision mAP is used as the main evaluation indicator of network performance, and the evaluation result pa is obtained. The calculation formula is: pa=mAP(Before test ,Gt test ) (22) Among them, mAP(*,*) is the calculation function of the average precision of yolo open source, Pred test and Gt test They respectively represent the three-dimensional target detection result set and the corresponding true value label test set.

Images