A processing method for adjusting point cloud target recognition results

Abstract

An embodiment of the present invention relates to a processing method for adjusting the point cloud target recognition result. The method includes: obtaining the first frame of point cloud at any time t; performing target recognition processing on the first frame of point cloud to obtain the corresponding point cloud target recognition result, and the point cloud target recognition result includes a plurality of first target recognition boxes Box i , each first target recognition box Box i corresponds to a set of first target recognition box parameters B i ; preprocessing the model input data according to the first frame of point cloud and the point cloud target recognition result to generate the corresponding first model input data; inputting the first model input data into the target box adjustment model for calculation to obtain the corresponding first model output data; the first model output data includes multiple sets of second target recognition box parameters B * i ; using each second target recognition box parameter B * i to replace the matching parameters in the first target recognition box parameter B i . Through the present invention, the target association accuracy of the perception module can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a processing method for adjusting point cloud target recognition results. Background Art

[0002] The perception module of the vehicle automatic driving system needs to perform target detection on the adjacent frame point cloud obtained by laser radar scanning to obtain multiple target detection frames on each frame, and implement state tracking of the target in the adjacent frame point cloud according to a set of target recognition frame parameters of each target detection frame (target detection frame center point coordinate Q, target detection frame length L, target detection frame width W, target detection frame height H, target detection frame orientation angle yaw, target detection frame type). When implementing state tracking, the conventional practice is to perform state prediction based on the target recognition frame parameters of the previous frame to obtain the corresponding predicted recognition frame parameters, and then use all the predicted recognition frame parameters of the previous frame and all the target recognition frame parameters of the next frame to associate the same target, and after the association relationship is determined, the motion trajectory of the target is constructed based on the previous and next two sets of target recognition frame parameters of the same target to achieve the purpose of state tracking. When associating the same target, the conventional practice is to perform association evaluation based on the similarity between the predicted recognition frame parameters of the previous frame and the target recognition frame parameters of the next frame, and take the correspondence between the pair of predicted recognition frame parameters of the previous frame and the target recognition frame parameters of the next frame with the highest similarity as the association basis for the corresponding target detection frames of the previous and next frames. It is not difficult to see that the target tracking accuracy of the perception module is dependent on the accuracy of parameters such as the position (center point coordinates), shape (length, width and height), orientation and type of the target recognition boxes in the previous and next frames. The accuracy of these parameters is in turn subject to the accuracy of the target detection model for point cloud target detection.

[0003] Conventional target detection models do not have temporal correlation features. Even when performing target detection on two frames of point clouds at adjacent moments, the target detection box parameters of the same target may change significantly. This change will directly lead to a large deviation in the similarity calculation, thereby causing problems such as target association misalignment or target association failure in the perception module. Summary of the invention

[0004] The purpose of the present invention is to provide a processing method, an electronic device and a computer-readable storage medium for adjusting the result of point cloud target recognition in view of the defects of the prior art, and to customize a target frame adjustment model with a gated recurrent unit (Gated Recurrent Unit, GRU) so that it can adjust the hidden state h transmitted from the previous time t-1 based on the hidden state h transmitted from the previous time t-1. t-1 And the point cloud target recognition result at the current time t calculates the hidden state h at the current time t t , and based on the hidden state h of the previous moment t-1The target recognition box parameters are adjusted for the point cloud target recognition result at the current time t, and the hidden state h at the current time t is t Continue to pass it down as the reference hidden state of the next moment. Through the present invention, based on the time series feature, the target recognition frame parameters in the point cloud target recognition result of the previous moment are adjusted with reference to the point cloud target recognition result of the next moment, which can make up for the defect that the output of the target detection model has no time series feature, thereby achieving the purpose of reducing the risk of target detection frame parameter jumps and improving the accuracy of target association of the perception module.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present invention provides a processing method for adjusting a point cloud target recognition result, the method comprising:

[0006] Get the first frame point cloud at any time t;

[0007] The first frame point cloud is processed for target recognition to obtain a corresponding point cloud target recognition result; the point cloud target recognition result includes a plurality of first target recognition boxes Box i ; Each of the first target recognition boxes Box i Corresponding to a set of first target recognition frame parameters B i ; i is the target recognition box index, 1≤i;

[0008] Performing model input data preprocessing according to the first frame point cloud and the point cloud target recognition result to generate corresponding first model input data;

[0009] The first model input data is input into the target frame adjustment model for calculation to obtain the corresponding first model output data; the first model output data includes multiple sets of second target recognition frame parameters The second target recognition frame parameter With the first target recognition frame parameter B i One to one correspondence;

[0010] Using the second target recognition box parameters The first target recognition frame parameter B i Replace the matching parameters in .

[0011] Preferably, the first target recognition frame parameter B i Including the first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ), the first length L i , the first width W i , first height H i , the first orientation angle yaw i and the first type Si ;

[0012] The second target recognition frame parameter Include the coordinates of the second center point Second length Second Width Second height and the second orientation angle

[0013] The first model input data includes a first point cloud coordinate group sequence, a first center point coordinate sequence, a first shape data group sequence, a first orientation angle sequence and a first type sequence;

[0014] The target frame adjustment model includes a feature extraction network, a backbone network and a regression head network; the feature extraction network is connected to the backbone network, and the backbone network is connected to the regression head network.

[0015] Preferably, the preprocessing of the model input data according to the first frame point cloud and the point cloud target recognition result to generate the corresponding first model input data specifically includes:

[0016] Each of the first target recognition boxes Box in the first frame point cloud i The point cloud coordinates of all points in the extract to form the corresponding first point cloud coordinate group P i , and obtain all the first point cloud coordinate groups P i The first point cloud coordinate group sequence is generated by sorting the target recognition frame index i; the first point cloud coordinate group P i Including multiple first point cloud coordinates p i,j (x i,j ,y i,j ,z i,j ), j is the point cloud coordinate index, 1≤j;

[0017] Set all the first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ) sorting the target recognition frame index i to generate the corresponding first center point coordinate sequence;

[0018] The first length L i , the first width W i and the first height H i The first shape data set D is formed i , and obtain all the first shape data sets D i Sorting according to the target recognition frame index i generates the corresponding first shape data group sequence;

[0019] Move all the first direction towards the angle yaw i Sorting the target recognition frame index i to generate the corresponding first orientation angle sequence;

[0020] All the first type S i Sorting the target recognition frame index i to generate the corresponding first type sequence;

[0021] The first point cloud coordinate group sequence, the first center point coordinate sequence, the first shape data group sequence, the first orientation angle sequence and the first type sequence obtained constitute the corresponding first model input data.

[0022] Preferably, the feature extraction network comprises a point cloud feature extraction unit, a center point feature extraction unit, a shape feature extraction unit, a direction angle feature extraction unit, a type feature extraction unit, a feature fusion unit and a fusion feature extraction unit; the point cloud feature extraction unit, the center point feature extraction unit, the shape feature extraction unit, the direction angle feature extraction unit and the type feature extraction unit are respectively connected to the feature fusion unit; the feature fusion unit is connected to the fusion feature extraction unit; the fusion feature extraction unit is connected to the backbone network;

[0023] The point cloud feature extraction unit is implemented based on the PointNet network, and the output vector data dimension of the point cloud feature extraction unit is 128; the center point feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the center point feature extraction unit is 32; the shape feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the shape feature extraction unit is 32; the orientation angle feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the orientation angle feature extraction unit is 32; the type feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the type feature extraction unit is 32; the output vector dimension of the feature fusion unit is 256; the fusion feature extraction unit is implemented based on the MLP network and includes two fully connected layers, and the output vector dimension of the fusion feature extraction unit is 128;

[0024] The backbone network is implemented based on a gated recurrent unit (GRU), and the output vector dimension of the backbone network is 256;

[0025] The regression head network includes a center point regression head unit, a shape regression head unit, and an orientation angle regression unit; the center point regression head unit, the shape regression head unit, and the orientation angle regression unit are respectively connected to the backbone network; the center point regression head unit includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the shape regression head unit includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the orientation angle regression unit includes two fully connected layers and there is a ReLU activation unit between the two fully connected layers.

[0026] Preferably, the step of inputting the first model input data into the target frame adjustment model to perform calculation to obtain the corresponding first model output data specifically includes:

[0027] Inputting the first model input data into the feature extraction network of the target frame adjustment model to perform feature extraction processing to generate a corresponding first feature vector;

[0028] The first feature vector is input into the backbone network of the target frame adjustment model, and the gated recurrent unit GRU is used to generate the hidden state h transmitted from the previous time t-1. t-1 And the first feature vector calculates the hidden state h at the current time t t ; and the hidden state h t As the hidden state to be transmitted to the next time t+1, the hidden state h t As the second feature vector output by the backbone network this time;

[0029] The second feature vector is input into the regression head network of the target box adjustment model to perform regression processing on the target box center point, target box shape and target box orientation angle to obtain the corresponding first model output data.

[0030] Furthermore, the step of inputting the first model input data into the feature extraction network of the target frame adjustment model to perform feature extraction processing to generate a corresponding first feature vector specifically includes:

[0031] Input the first point cloud coordinate group sequence of the first model input data into the point cloud feature extraction unit of the feature extraction network to perform point cloud feature extraction processing to generate a corresponding first point cloud feature vector; the data dimension of the first point cloud feature vector is 128;

[0032] Input the first center point coordinate sequence of the first model input data into the center point feature extraction unit of the feature extraction network to perform center point feature vector conversion processing to generate a corresponding first center point feature vector; the data dimension of the first center point feature vector is 32;

[0033] Input the first shape data group sequence of the first model input data into the shape feature extraction unit of the feature extraction network to perform shape feature vector conversion processing to generate a corresponding first shape feature vector; the data dimension of the first shape feature vector is 32;

[0034] Input the first orientation angle sequence of the first model input data into the orientation angle feature extraction unit of the feature extraction network to perform orientation angle feature vector conversion processing to generate a corresponding first orientation angle feature vector; the data dimension of the first orientation angle feature vector is 32;

[0035] Input the first type sequence of the first model input data into the type feature extraction unit of the feature extraction network to perform type feature vector conversion processing to generate a corresponding first type feature vector; the data dimension of the first type feature vector is 32;

[0036] Inputting the first point cloud feature vector, the first center point feature vector, the first shape feature vector, the first orientation angle feature vector and the first type feature vector into the feature fusion unit of the feature extraction network for vector splicing processing to generate a corresponding first splicing vector; the data dimension of the first splicing vector is 256;

[0037] The first concatenated vector is input into the fusion feature extraction unit of the feature extraction network to perform two full-connection calculations to generate the corresponding first feature vector; the data dimension of the first feature vector is 128.

[0038] Furthermore, the gated recurrent unit GRU is based on the hidden state h transmitted from the previous time t-1. t-1 And the first feature vector calculates the hidden state h at the current time t t , including:

[0039] The gated recurrent unit GRU is based on the hidden state h t-1 The reset gate state r at the current time t is calculated by using the first feature vector t and update the gate state z t ; and based on the reset door state r t , the hidden state h t-1 And the first feature vector calculates the candidate hidden state at the current time t And update the door state z according to t , the hidden state h t-1 and the candidate hidden state Calculate the hidden state h at the current time t t .

[0040] Further, the step of inputting the second feature vector into the regression head network of the target frame adjustment model to perform regression processing on the target frame center point, target frame shape and target frame orientation angle to obtain the corresponding first model output data specifically includes:

[0041] The second feature vector is input into the center point regression head unit of the regression head network, and the corresponding first center point regression vector is obtained after two full connection calculations with ReLU activation processing and then one full connection processing; the first center point regression vector is composed of multiple coordinates of the second center point composition;

[0042] The second feature vector is input into the shape regression head unit of the regression head network, and the corresponding first shape regression vector is obtained after two full connection calculations with ReLU activation processing and then one full connection processing; the first shape regression vector is composed of multiple second shape data groups Composition: each of the second shape data sets Includes a set of the second length The second width and the second height

[0043] The second feature vector is input into the orientation angle regression unit of the regression head network, and the corresponding first orientation angle regression vector is obtained after a full connection calculation with ReLU activation processing and then a full connection processing; the first orientation angle regression vector is composed of multiple second orientation angles. composition;

[0044] The second center point coordinates corresponding to each target recognition frame index i are The second length The second width The second height and the second orientation angle The corresponding second target recognition frame parameters And all the second target recognition frame parameters obtained are The first model output data is formed.

[0045] A second aspect of an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0046] The processor is used to be coupled to the memory, read and execute instructions in the memory, so as to implement the method steps described in the first aspect above;

[0047] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0048] A third aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed by a computer, the computer executes the instructions of the method described in the first aspect above.

[0049] The embodiment of the present invention provides a processing method, an electronic device and a computer-readable storage medium for adjusting the result of point cloud target recognition, and customizes a target frame adjustment model with GRU so that it can adjust the hidden state h transmitted from the previous time t-1. t-1 And the point cloud target recognition result at the current time t calculates the hidden state h at the current time t t , and based on the hidden state h of the previous moment t-1 The target recognition box parameters are adjusted for the point cloud target recognition result at the current time t, and the hidden state h at the current time t is t Continue to pass it down as the reference hidden state of the next moment. Through the present invention, the target recognition frame parameters in the point cloud target recognition result of the next moment can be adjusted based on the temporal characteristics with reference to the point cloud target recognition result of the previous moment, thereby compensating for the defect that the output of the target detection model has no temporal characteristics, reducing the risk of jump changes in the target detection frame parameters, and improving the accuracy of target association of the perception module. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a processing method for adjusting point cloud target recognition results provided in Embodiment 1 of the present invention;

[0051] Figure 2 A schematic diagram of the module structure of the target frame adjustment model provided in the first embodiment of the present invention;

[0052] Figure 3 A schematic diagram of the structure of an electronic device provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment 1 of the present invention provides a processing method for adjusting a point cloud target recognition result, such as Figure 1As shown in a schematic diagram of a processing method for adjusting a point cloud target recognition result provided in the first embodiment of the present invention, the method mainly comprises the following steps:

[0055] Step 1: Get the first frame of point cloud at any time t.

[0056] Here, the first frame of point cloud is the point cloud data obtained by the perception module from the lidar.

[0057] Step 2, performing target recognition processing on the first frame point cloud to obtain the corresponding point cloud target recognition result;

[0058] Among them, the point cloud target recognition result includes multiple first target recognition boxes Box i ; Each first target recognition box Box i Corresponding to a set of first target recognition frame parameters B i , i is the target recognition frame index, 1≤i; the first target recognition frame parameter B i Including the first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ), the first length L i , the first width W i , first height H i , the first orientation angle yaw i and the first type S i ; Type 1 S i At least include pedestrian type, vehicle type and bicycle type.

[0059] Here, target recognition processing for the first frame of point cloud actually uses the target detection model to perform point cloud target recognition on the first frame of point cloud. The target detection model can adopt PointPillars model, VoxelNet model, PointNet model, etc. The technical implementation of each model can be obtained from the public technical solution, which will not be described one by one here; it should be noted that these models do not have the characteristics of time series processing, that is, the point cloud target detection at any moment will not refer to the point cloud target recognition result of the previous moment as a reference, so the point cloud target recognition results of the previous and subsequent moments are bound to have the risk of jumps in the target detection box results. The point cloud target recognition results output by each model can be regarded as consisting of multiple three-dimensional target recognition boxes, that is, the first target recognition box Box i Each first target recognition box Box i Corresponding to a set of parameters that identify the position, shape, orientation and type of the target recognition frame, that is, the first target recognition frame parameters B i ; In the first target recognition frame parameter B i The coordinates of the first center point in i (xQ,i ,y Q,i ,z Q,i ) is the center point position parameter of the current target recognition frame, the first length L i , the first width W i , first height H i is the shape parameter of the current target recognition box, the first orientation angle yaw i is the orientation parameter of the current target recognition box, the first type S i The target type parameters of the current target recognition box, such as pedestrian type, vehicle type, and bicycle type.

[0060] It should be noted that if the target detection model outputs not a three-dimensional target recognition box, but a regular or irregular convex polygon on a bird's-eye view plane based on a bird's-eye view (BEV), then the embodiment of the present invention further includes: performing a closest rectangular frame drawing process on each convex polygon obtained during the target recognition process to obtain a plurality of corresponding regular rectangles, and using each regular rectangle as the corresponding first target recognition box Box i ; and extract the plane coordinates (x, y) of the center points of each regular rectangle as the corresponding first target recognition frame parameters B i The first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ) of (x Q,i ,y Q,i ) and the coordinates of the first center point Q i z Q,i Set to 0; and extract the length and width of each regular rectangle as the corresponding first target recognition frame parameter B i The first length L i , the first width W i , and the first height H i Set to 0; and extract the orientation parameters corresponding to each regular rectangle as the corresponding first target recognition frame parameters B i The first heading angle yaw i ; and extract the target type parameters corresponding to each regular rectangle as the corresponding first target recognition frame parameters B i The first type S i In this way, multiple sets of first target recognition frame parameters B in the same data format as the above data can also be obtained. i .

[0061] Step 3, preprocessing the model input data according to the first frame point cloud and the point cloud target recognition result to generate corresponding first model input data;

[0062] The first model input data includes a first point cloud coordinate group sequence, a first center point coordinate sequence, a first shape data group sequence, a first orientation angle sequence and a first type sequence;

[0063] Here, actually, the input data required for the target frame adjustment model is prepared before the point cloud target recognition result at the current moment is adjusted into the target frame parameters;

[0064] Specifically, step 31 includes: each first target recognition box Box in the first frame point cloud i The point cloud coordinates of all points in the extract to form the corresponding first point cloud coordinate group P i , and obtain all the first point cloud coordinate groups P i Sorting by target recognition frame index i generates a corresponding first point cloud coordinate group sequence;

[0065] Among them, the first point cloud coordinate group P i Including multiple first point cloud coordinates p i,j (x i,j ,y i,j ,z i,j ), j is the point cloud coordinate index, 1≤j;

[0066] Here, based on the known first target recognition frame parameter B i The first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ), the first length L i , the first width W i and the first height H i , we can get each first target recognition box Box i In the spatial range of the point cloud space, the point cloud coordinates in the first frame point cloud are in each first target recognition box Box i The point cloud coordinates of the points in the spatial range can be extracted to obtain the first target recognition box Box i The corresponding first point cloud coordinate group P i ; Each first point cloud coordinate group P i It is composed of the point cloud coordinates of multiple points. The point cloud coordinates of each point are the first point cloud coordinates p i,j (x i,j ,y i,j ,z i,j ), the subscript index (i, j) represents the current first point cloud coordinate p i,j Specifically, it is the point cloud coordinates of the jth point of the i-th first target recognition frame; after obtaining the coordinates of the jth point of each first target recognition frame Box i The corresponding first point cloud coordinate group P iAfterwards, the first point cloud coordinate group P is mapped according to the target recognition frame index i i Sorting can get the first point cloud coordinate group sequence;

[0067] Step 32: Set all first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ) sorting by target recognition frame index i to generate a corresponding first center point coordinate sequence;

[0068] Here, the first center point coordinate Q in the first center point coordinate sequence i (x Q,i ,y Q,i ,z Q,i ) is consistent with the sorting order in the first point cloud coordinate group sequence;

[0069] Step 33, the first length L i , the first width W i and the first height H i The first shape data set D is composed of i , and all the first shape data sets D i Sorting by target recognition frame index i generates a corresponding first shape data group sequence;

[0070] Here, each first target recognition box Box i The shape parameter of is actually composed of three parameters: length, width, and height. Therefore, the first length L corresponding to the same target recognition box index i i , the first width W i and the first height H i A first shape data set D is formed i , the first shape data set D in the first shape data set sequence i The sorting order of is also consistent with the sorting order in the first point cloud coordinate group sequence;

[0071] Step 34: Yaw all the first facing angles i Sort by target recognition frame index i to generate a corresponding first orientation angle sequence;

[0072] Here, the first orientation angle yaw in the first orientation angle sequence i The sorting order of is also consistent with the sorting order in the first point cloud coordinate group sequence;

[0073] Step 35: All first type S i Sort by target recognition frame index i to generate a corresponding first type sequence;

[0074] Here, the first type S in the first type sequencei The sorting order of is also consistent with the sorting order in the first point cloud coordinate group sequence;

[0075] Step 36, the obtained first point cloud coordinate group sequence, first center point coordinate sequence, first shape data group sequence, first orientation angle sequence and first type sequence form the corresponding first model input data.

[0076] Here, it is not difficult to see that when preparing the model input data, all the first target recognition frame parameters B in the point cloud target recognition results are actually classified into five categories: point cloud set in the target recognition frame, target recognition frame position, target recognition frame shape, target recognition frame orientation and target recognition frame type. i The parameters are classified to obtain five types of original data, namely: the first point cloud coordinate group sequence, the first center point coordinate sequence, the first shape data group sequence, the first orientation angle sequence and the first type sequence.

[0077] Step 4, inputting the first model input data into the target frame adjustment model for calculation to obtain the corresponding first model output data;

[0078] The first model output data includes multiple sets of second target recognition frame parameters Second target recognition frame parameters and the first target recognition frame parameter B i One-to-one correspondence; second target recognition frame parameters Include the coordinates of the second center point Second length Second Width Second height and the second orientation angle

[0079] Here, the first model output data is actually the first target recognition frame parameters B i After the correction data is modulated by the model, the first model output data is also composed of multiple sets of target recognition frame parameters, that is, the second target recognition frame parameters The second target recognition frame parameters output by the model of the embodiment of the present invention are constituted The number and first target recognition box parameter B i The number of is the same, so the second target recognition frame parameters can be formed and the first target recognition frame parameter B i One-to-one correspondence; each second target recognition box parameter The parameters of the first target recognition box parameter B i Similarly, the coordinates of the second center point are also included Second length Second Width Second height and the second orientation angle But it does not include the type parameter, because the first and second target recognition frame parameters correspond one to one, so the type does not need to be adjusted.

[0080] The target box adjustment model used in the step is as follows Figure 2 As shown in the module structure diagram of the target frame adjustment model provided in the first embodiment of the present invention, it includes a feature extraction network 201, a backbone network 202 and a regression head network 203; the feature extraction network 201 is connected to the backbone network 202, and the backbone network 202 is connected to the regression head network 203.

[0081] Among them, the feature extraction network 201 includes a point cloud feature extraction unit 2011, a center point feature extraction unit 2012, a shape feature extraction unit 2013, an orientation angle feature extraction unit 2014, a type feature extraction unit 2015, a feature fusion unit 2016 and a fusion feature extraction unit 2017; the point cloud feature extraction unit 2011, the center point feature extraction unit 2012, the shape feature extraction unit 2013, the orientation angle feature extraction unit 2014 and the type feature extraction unit 2015 are respectively connected to the feature fusion unit 2016; the feature fusion unit 2016 is connected to the fusion feature extraction unit 2017; the fusion feature extraction unit 2017 is connected to the backbone network 202;

[0082] The point cloud feature extraction unit 2011 is implemented based on the PointNet network, and the output vector data dimension of the point cloud feature extraction unit 2011 is 128; the center point feature extraction unit 2012 is based on the Multilayer Perception (Multilayer The output vector dimension of the center point feature extraction unit 2012 is 32; the shape feature extraction unit 2013 is based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the shape feature extraction unit 2013 is 32; the orientation angle feature extraction unit 2014 is based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the orientation angle feature extraction unit 2014 is 32; the type feature extraction unit 2015 is based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the type feature extraction unit 2015 is 32; the output vector dimension of the feature fusion unit 2016 is 256; the fusion feature extraction unit 2017 is based on the MLP network and includes two fully connected layers, and the output vector dimension of the fusion feature extraction unit 2017 is 128;

[0083] The backbone network 202 is implemented based on the gated recurrent unit GRU, and the output vector dimension of the backbone network 202 is 256;

[0084] The regression head network 203 includes a center point regression head unit 2031, a shape regression head unit 2032, and a direction angle regression unit 2033; the center point regression head unit 2031, the shape regression head unit 2032, and the direction angle regression unit 2033 are respectively connected to the backbone network 202;

[0085] The center point regression head unit 2031 includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the shape regression head unit 2032 includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the orientation angle regression unit 2033 includes two fully connected layers and there is a ReLU activation unit between the two fully connected layers.

[0086] Based on the above target frame adjustment model, step 4 inputs the first model input data into the target frame adjustment model for calculation to obtain the corresponding first model output data, specifically including:

[0087] Step 41, inputting the first model input data into the feature extraction network 201 of the target frame adjustment model to perform feature extraction processing to generate a corresponding first feature vector;

[0088] Here, in fact, based on the feature extraction network 201, the five types of original data in the first model input data (the first point cloud coordinate group sequence, the first center point coordinate sequence, the first shape data group sequence, the first orientation angle sequence and the first type sequence) are respectively subjected to feature extraction;

[0089] Specifically, it includes: step 411, inputting the first point cloud coordinate group sequence of the first model input data into the point cloud feature extraction unit 211 of the feature extraction network 201 to perform point cloud feature extraction processing to generate a corresponding first point cloud feature vector;

[0090] Among them, the data dimension of the first point cloud feature vector is 128;

[0091] Here, the point cloud feature extraction unit 2011 is implemented with reference to the PointNet network. For the network structure of the PointNet network, please refer to the paper "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation", which will not be further described here. As can be seen from the paper, the PointNet network has two application variations: a classification network application for classification and a segmentation network application for segmentation. Both application front ends use the same multi-layer convolutional network as the backbone network for feature extraction. The difference between the two applications is that the classification network application uses an MLP network to classify the feature tensor with a dimension of 1024 output by the backbone network to obtain multiple classification results; while the segmentation network application first uses a global feature and a feature tensor with a dimension of 1024 output by the backbone network to perform feature fusion to obtain a fusion tensor with a dimension of 1088, and then uses a fusion tensor with three layers of output dimensions of An MLP network composed of fully connected layers of 512, 256, and 128 performs feature extraction on the fused tensor to obtain a feature vector with a dimension of 128, and then uses an MLP network composed of two layers of fully connected layers with output dimensions of 128 and m respectively to perform semantic segmentation processing on the feature vector to obtain a semantic segmentation map with a shape of m*m; It should be noted that the point cloud feature extraction unit 2011 of the embodiment of the present invention is specifically implemented based on the segmentation network application of the PointNet network, and extracts the feature vector with a dimension of 128 output by the MLP network composed of three layers of fully connected layers with output dimensions of 512, 256, and 128 respectively as the corresponding first point cloud feature vector;

[0092] Step 412, inputting the first center point coordinate sequence of the first model input data into the center point feature extraction unit 2012 of the feature extraction network 201 to perform center point feature vector conversion processing to generate a corresponding first center point feature vector;

[0093] Among them, the data dimension of the first center point feature vector is 32;

[0094] Here, when the center point feature extraction unit 2012 performs the center point feature vector conversion processing, it actually completes the feature extraction of the first center point coordinate sequence through a fully connected layer and a ReLU activation unit to obtain the first center point feature vector; as can be seen from the foregoing, the output vector dimension of the center point feature extraction unit 2012 is specified to be 32 when the model is designed, so the data dimension of the first center point feature vector is 32;

[0095] Step 413, inputting the first shape data group sequence of the first model input data into the shape feature extraction unit 2013 of the feature extraction network 201 to perform shape feature vector conversion processing to generate a corresponding first shape feature vector;

[0096] Among them, the data dimension of the first shape feature vector is 32;

[0097] Here, when the shape feature extraction unit 2013 performs the shape feature vector conversion processing, it actually completes the feature extraction of the first shape data group sequence through a fully connected layer and a ReLU activation unit to obtain the first shape feature vector; as can be seen from the above, the output vector dimension of the shape feature extraction unit 2013 is specified to be 32 when the model is designed, so the data dimension of the first shape feature vector is 32;

[0098] Step 414, inputting the first orientation angle sequence of the first model input data into the orientation angle feature extraction unit 2014 of the feature extraction network 201 to perform orientation angle feature vector conversion processing to generate a corresponding first orientation angle feature vector;

[0099] Among them, the data dimension of the first orientation angle feature vector is 32;

[0100] Here, when the orientation angle feature extraction unit 2014 performs the orientation angle feature vector conversion processing, it actually completes the feature extraction of the first orientation angle sequence through a fully connected layer and a ReLU activation unit to obtain the first orientation angle feature vector; as can be seen from the foregoing, the output vector dimension of the orientation angle feature extraction unit 2014 is specified to be 32 during model design, so the data dimension of the first orientation angle feature vector is 32;

[0101] Step 415, inputting the first type sequence of the first model input data into the type feature extraction unit 2015 of the feature extraction network 201 to perform type feature vector conversion processing to generate a corresponding first type feature vector;

[0102] Among them, the data dimension of the first type of feature vector is 32;

[0103] Here, when the type feature extraction unit 2015 performs the type feature vector conversion processing, it actually completes the feature extraction of the first type sequence through a fully connected layer and a ReLU activation unit to obtain the first type feature vector; as can be seen from the above, the output vector dimension of the type feature extraction unit 2015 is specified to be 32 when the model is designed, so the data dimension of the first type feature vector is 32;

[0104] Step 416, inputting the first point cloud feature vector, the first center point feature vector, the first shape feature vector, the first orientation angle feature vector and the first type feature vector into the feature fusion unit 2016 of the feature extraction network 201 to perform vector splicing processing to generate a corresponding first splicing vector;

[0105] Among them, the data dimension of the first splicing vector is 256;

[0106] Here, the feature fusion unit 2016 actually concatenates the five feature vectors obtained in steps 411-415, so the dimension of the output concatenated vector = 128 + 32*4 = 256;

[0107] Step 417, input the first concatenated vector into the fusion feature extraction unit 2017 of the feature extraction network 201 to perform two full connection calculations to generate a corresponding first feature vector;

[0108] Among them, the data dimension of the first eigenvector is 128;

[0109] Here, the fusion feature extraction unit 2017 performs two full connection calculations on the first splicing vector based on its own two fully connected layers to complete the feature extraction operation on the first splicing vector; as can be seen from the foregoing, the output vector dimension of the fusion feature extraction unit 2017 is specified to be 128 when the model is designed, so the data dimension of the first feature vector is 128;

[0110] Step 42: input the first feature vector into the backbone network 202 of the target frame adjustment model, and pass it through the GRU based on the hidden state h transmitted at the previous time t-1. t-1 And the first eigenvector calculates the hidden state h at the current time t t ; and hide the state h t As the hidden state passed to the next time t+1, the hidden state h t As the second feature vector output by the backbone network 202 this time;

[0111] Specifically include: through GRU based on the hidden state h t-1 And the first eigenvector calculates the reset gate state r at the current time t t and update the gate state z t ; and based on the reset gate state r t , hidden state h t-1 And the first eigenvector calculates the candidate hidden state at the current time t And update the door state z according to t , hidden state h t-1 and candidate hidden states Calculate the hidden state h at the current time t t ;

[0112] Here, the specific calculation method of the gated recurrent unit, also known as GRU, can be referred to the paper "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling", which will not be further elaborated here; let the first eigenvector be x in the paper t , it can be seen from the paper that the calculation steps of GRU are based on h t-1 、x t First calculate r t 、z t , then based on r t 、h t-1 and x t work out Then based on h t-1 and z t Calculate h t ; According to the paper, each time GRU calculates, the output y t Actually it is h t ; Therefore, in getting h t After that, it should not only be passed down as the hidden state of the next moment (t+1), but also used as the y t The output is used as the second eigenvector. It is not difficult to see from the calculation process of the paper that the second eigenvector is actually an adjustment of the first eigenvector corresponding to the point cloud target recognition result at the previous moment by referring to the second eigenvector corresponding to the point cloud target recognition result at the previous moment. This calculation method of referring to the hidden state adds a time series feature to the second eigenvector of this time.

[0113] Step 43, inputting the second feature vector into the regression head network 203 of the target frame adjustment model to perform regression processing on the target frame center point, target frame shape and target frame orientation angle to obtain corresponding first model output data;

[0114] Specifically, the step 431 includes: inputting the second feature vector into the center point regression head unit 2031 of the regression head network 203, performing two full connection calculations with ReLU activation processing and then performing one full connection processing to obtain a corresponding first center point regression vector;

[0115] Among them, the first center point regression vector is composed of multiple second center point coordinates composition;

[0116] Here, the center point regression head unit 2031 performs two full connection calculations with ReLU activation processing and then one full connection processing, extracts the features related to the center point position from the second feature vector for regression calculation to obtain the adjusted first target recognition boxes Boxi The center point position is also the second center point coordinate

[0117] Step 432, input the second feature vector into the shape regression head unit 2032 of the regression head network 203, perform two full connection calculations with ReLU activation processing and then perform one full connection processing to obtain the corresponding first shape regression vector;

[0118] The first shape regression vector is composed of a plurality of second shape data sets Composition: Each second shape data group Includes a set of second lengths Second Width and the second height

[0119] Here, the shape regression head unit 2032 performs two full connection calculations with ReLU activation processing and then one full connection processing, extracts the features related to the shape (length, width and height) from the second feature vector for regression calculation to obtain the adjusted first target recognition boxes Box i The shape data of the second shape data group Each second shape data set It is composed of a set of length, width and height parameters, which is the second length Second Width and the second height

[0120] Step 433, input the second feature vector into the orientation angle regression unit 2033 of the regression head network 203, and obtain the corresponding first orientation angle regression vector after a full connection calculation with ReLU activation processing and another full connection processing;

[0121] The first orientation angle regression vector is composed of multiple second orientation angles composition;

[0122] Here, the orientation angle regression unit 2033 performs a full connection calculation with ReLU activation processing and then a full connection processing, extracts the features related to the orientation angle from the second feature vector for regression calculation to obtain the adjusted first target recognition boxes Box i The orientation angle is also called the second orientation angle

[0123] Step 434: The second center point coordinates corresponding to each target recognition frame index i are Second length Second Width Second height and the second orientation angle The corresponding second target recognition frame parameters And all the second target recognition box parameters obtained are The first model output data is composed.

[0124] Step 5: Use the parameters of each second target recognition box Parameter B for the first target recognition frame i Replace the matching parameters in ;

[0125] Specifically, it includes: according to the corresponding relationship of the target frame index i, using the second target recognition frame parameter The coordinates of the second center point in The corresponding first target recognition frame parameter B i The first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ) is replaced; using the second target recognition frame parameters The second length in Second Width Second height The corresponding first target recognition frame parameter B i The first length L i , the first width W i , first height H i Replace; use the second target recognition frame parameters The second orientation angle The corresponding first target recognition frame parameter B i The first heading angle yaw i Make a replacement.

[0126] Figure 3 This is a schematic diagram of the structure of an electronic device provided in the second embodiment of the present invention. The electronic device may be the aforementioned terminal device or server, or may be a terminal device or server connected to the aforementioned terminal device or server to implement the method of the embodiment of the present invention. Figure 3 As shown, the electronic device may include: a processor 301 (such as a CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transceiver 303. Various instructions may be stored in the memory 302 to complete various processing functions and implement the processing steps described in the aforementioned method embodiment. Preferably, the electronic device involved in the embodiment of the present invention also includes: a power supply 304, a system bus 305 and a communication port 306. The system bus 305 is used to realize the communication connection between components. The above-mentioned communication port 306 is used for connecting and communicating between the electronic device and other peripherals.

[0127] exist Figure 3 The system bus 305 mentioned in the figure can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The term "communication interface" is represented by only one thick line, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory (Non-Volatile Memory), such as at least one disk storage.

[0128] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a graphics processing unit (GPU), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0129] It should be noted that an embodiment of the present invention further provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method and processing process provided in the above embodiments.

[0130] An embodiment of the present invention further provides a chip for executing instructions, wherein the chip is used to execute the processing steps described in the aforementioned method embodiment.

[0131] The embodiment of the present invention provides a processing method, an electronic device and a computer-readable storage medium for adjusting the result of point cloud target recognition, and customizes a target frame adjustment model with GRU so that it can adjust the hidden state h transmitted from the previous time t-1. t-1 And the point cloud target recognition result at the current time t calculates the hidden state h at the current time t t, and based on the hidden state h of the previous moment t-1 The target recognition box parameters are adjusted for the point cloud target recognition result at the current time t, and the hidden state h at the current time t is t Continue to pass it down as the reference hidden state of the next moment. Through the present invention, the target recognition frame parameters in the point cloud target recognition result of the next moment can be adjusted based on the temporal characteristics with reference to the point cloud target recognition result of the previous moment, thereby compensating for the defect that the output of the target detection model has no temporal characteristics, reducing the risk of jump changes in the target detection frame parameters, and improving the accuracy of target association of the perception module.

[0132] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0133] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0134] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A processing method for adjusting point cloud target recognition results, characterized in that: The method comprises: Get the first frame point cloud at any time t; The first frame point cloud is processed for target recognition to obtain a corresponding point cloud target recognition result; the point cloud target recognition result includes a plurality of first target recognition boxes Box i ; Each of the first target recognition boxes Box i Corresponding to a set of first target recognition frame parameters B i ; i is the target recognition box index, 1≤i; Performing model input data preprocessing according to the first frame point cloud and the point cloud target recognition result to generate corresponding first model input data; The first model input data is input into the target frame adjustment model for calculation to obtain the corresponding first model output data; the first model output data includes multiple sets of second target recognition frame parameters The second target recognition frame parameter With the first target recognition frame parameter B i One to one correspondence; Using the second target recognition box parameters The first target recognition frame parameter B i Replace the matching parameters in; Among them, the first target recognition frame parameter B i Including the first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ), the first length L i , the first width W i , first height H i , the first orientation angle yaw i and the first type S i ; The second target recognition frame parameter Include the coordinates of the second center point Second length Second Width Second height and the second orientation angle The first model input data includes a first point cloud coordinate group sequence, a first center point coordinate sequence, a first shape data group sequence, a first orientation angle sequence and a first type sequence; The target frame adjustment model includes a feature extraction network, a backbone network and a regression head network; the feature extraction network is connected to the backbone network, and the backbone network is connected to the regression head network; The feature extraction network comprises a point cloud feature extraction unit, a center point feature extraction unit, a shape feature extraction unit, a direction angle feature extraction unit, a type feature extraction unit, a feature fusion unit and a fusion feature extraction unit; the point cloud feature extraction unit, the center point feature extraction unit, the shape feature extraction unit, the direction angle feature extraction unit and the type feature extraction unit are respectively connected to the feature fusion unit; the feature fusion unit is connected to the fusion feature extraction unit; the fusion feature extraction unit is connected to the backbone network; The point cloud feature extraction unit is implemented based on the PointNet network, and the output vector data dimension of the point cloud feature extraction unit is 128; the center point feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the center point feature extraction unit is 32; the shape feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the shape feature extraction unit is 32; the orientation angle feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the orientation angle feature extraction unit is 32; the type feature extraction unit is implemented based on the MLP network and includes a fully connected layer and a ReLU activation unit, and the output vector dimension of the type feature extraction unit is 32; the output vector dimension of the feature fusion unit is 256; the fusion feature extraction unit is implemented based on the MLP network and includes two fully connected layers, and the output vector dimension of the fusion feature extraction unit is 128; The backbone network is implemented based on a gated recurrent unit (GRU), and the output vector dimension of the backbone network is 256; The regression head network includes a center point regression head unit, a shape regression head unit, and an orientation angle regression unit; the center point regression head unit, the shape regression head unit, and the orientation angle regression unit are respectively connected to the backbone network; the center point regression head unit includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the shape regression head unit includes three fully connected layers and there is a ReLU activation unit between two adjacent fully connected layers; the orientation angle regression unit includes two fully connected layers and there is a ReLU activation unit between the two fully connected layers; The step of inputting the first model input data into the target frame adjustment model to perform calculation to obtain corresponding first model output data specifically includes: Inputting the first model input data into the feature extraction network of the target frame adjustment model to perform feature extraction processing to generate a corresponding first feature vector; The first feature vector is input into the backbone network of the target frame adjustment model, and the gated recurrent unit GRU is used to generate the hidden state h transmitted from the previous time t-1. t-1 And the first feature vector calculates the hidden state h at the current time t t ; and the hidden state h t As the hidden state to be transmitted to the next time t+1, the hidden state h t As the second feature vector output by the backbone network this time; The second feature vector is input into the regression head network of the target box adjustment model to perform regression processing on the target box center point, target box shape and target box orientation angle to obtain the corresponding first model output data.

2. The method for adjusting the point cloud target recognition result according to claim 1, characterized in that: The preprocessing of the model input data according to the first frame point cloud and the point cloud target recognition result to generate the corresponding first model input data specifically includes: Each of the first target recognition boxes Box in the first frame point cloud i The point cloud coordinates of all points in the extract to form the corresponding first point cloud coordinate group P i , and obtain all the first point cloud coordinate groups P i The first point cloud coordinate group sequence is generated by sorting the target recognition frame index i; the first point cloud coordinate group P i Including multiple first point cloud coordinates p i,j (x i,j ,y i,j ,z i,j ), j is the point cloud coordinate index, 1≤j; Set all the first center point coordinates Q i (x Q,i ,y Q,i ,z Q,i ) sorting the target recognition frame index i to generate the corresponding first center point coordinate sequence; The first length L i , the first width W i and the first height H i The first shape data set D is composed of i , and obtain all the first shape data sets D i Sorting according to the target recognition frame index i generates the corresponding first shape data group sequence; Move all the first direction toward the angle yaw i Sorting the target recognition frame index i to generate the corresponding first orientation angle sequence; All the first type S i Sorting the target recognition frame index i to generate the corresponding first type sequence; The first point cloud coordinate group sequence, the first center point coordinate sequence, the first shape data group sequence, the first orientation angle sequence and the first type sequence obtained constitute the corresponding first model input data.

3. The method for adjusting the point cloud target recognition result according to claim 1, characterized in that: The step of inputting the first model input data into the feature extraction network of the target frame adjustment model to perform feature extraction processing to generate a corresponding first feature vector specifically includes: Input the first point cloud coordinate group sequence of the first model input data into the point cloud feature extraction unit of the feature extraction network to perform point cloud feature extraction processing to generate a corresponding first point cloud feature vector; the data dimension of the first point cloud feature vector is 128; Input the first center point coordinate sequence of the first model input data into the center point feature extraction unit of the feature extraction network to perform center point feature vector conversion processing to generate a corresponding first center point feature vector; the data dimension of the first center point feature vector is 32; Input the first shape data group sequence of the first model input data into the shape feature extraction unit of the feature extraction network to perform shape feature vector conversion processing to generate a corresponding first shape feature vector; the data dimension of the first shape feature vector is 32; Input the first orientation angle sequence of the first model input data into the orientation angle feature extraction unit of the feature extraction network to perform orientation angle feature vector conversion processing to generate a corresponding first orientation angle feature vector; the data dimension of the first orientation angle feature vector is 32; Input the first type sequence of the first model input data into the type feature extraction unit of the feature extraction network to perform type feature vector conversion processing to generate a corresponding first type feature vector; the data dimension of the first type feature vector is 32; Inputting the first point cloud feature vector, the first center point feature vector, the first shape feature vector, the first orientation angle feature vector and the first type feature vector into the feature fusion unit of the feature extraction network for vector splicing processing to generate a corresponding first splicing vector; the data dimension of the first splicing vector is 256; The first concatenated vector is input into the fusion feature extraction unit of the feature extraction network to perform two full-connection calculations to generate the corresponding first feature vector; the data dimension of the first feature vector is 128.

4. The method for adjusting the point cloud target recognition result according to claim 1, characterized in that: The gated recurrent unit GRU is based on the hidden state h transmitted at the previous time t-1 t-1 And the first feature vector calculates the hidden state h at the current time t t , specifically including: The gated recurrent unit GRU is based on the hidden state h t-1 The reset gate state r at the current time t is calculated by using the first feature vector t and update the gate state z t ; and based on the reset door state r t , the hidden state h t-1 And the first feature vector calculates the candidate hidden state at the current time t And update the door state z according to t , the hidden state h t-1 and the candidate hidden state Calculate the hidden state h at the current time t t .

5. The method for adjusting the point cloud target recognition result according to claim 1, characterized in that: The step of inputting the second feature vector into the regression head network of the target frame adjustment model to perform regression processing on the target frame center point, the target frame shape and the target frame orientation angle to obtain the corresponding first model output data specifically includes: The second feature vector is input into the center point regression head unit of the regression head network, and the corresponding first center point regression vector is obtained after two full connection calculations with ReLU activation processing and then one full connection processing; the first center point regression vector is composed of multiple coordinates of the second center point composition; The second feature vector is input into the shape regression head unit of the regression head network, and the corresponding first shape regression vector is obtained after two full connection calculations with ReLU activation processing and then one full connection processing; the first shape regression vector is composed of multiple second shape data groups Composition: each of the second shape data sets Includes a set of the second length The second width and the second height The second feature vector is input into the orientation angle regression unit of the regression head network, and the corresponding first orientation angle regression vector is obtained after a full connection calculation with ReLU activation processing and then a full connection processing; the first orientation angle regression vector is composed of multiple second orientation angles. composition; The second center point coordinates corresponding to each target recognition frame index i are The second length The second width The second height and the second orientation angle The corresponding second target recognition frame parameters And all the second target recognition frame parameters obtained are The first model output data is formed.

6. An electronic device, characterized in that: include: memory, processors, and transceivers; The processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method steps described in any one of claims 1 to 5; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when executed by a computer, enable the computer to execute the method according to any one of claims 1 to 5.

Images