Three-dimensional target detection model training method, three-dimensional target detection method and device

Abstract

The present application provides a training method for a three-dimensional target detection model, a three-dimensional target detection method and an apparatus, the training method comprising obtaining a training data set; performing voxelization processing on the point cloud training data based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features; using a first initial coding network to process the plurality of grid point cloud features to obtain target point cloud features, wherein the first initial coding network is constructed based on a serialization module and a target linear complexity model; performing target detection processing on the target point cloud features using a second initial coding network to obtain a target prediction result; performing matching processing on the target prediction result and the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result; inputting the target matching result into a target loss function to obtain a target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model.

Description

Technical Field

[0001] The present application relates to the field of computer vision and neural network technology, and more specifically, to a three-dimensional target detection model training method, a three-dimensional target detection method, a three-dimensional target detection model training device, a three-dimensional target detection device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] 3D object detection is a key technology in computer vision, aiming to accurately identify and locate objects in three-dimensional space using point cloud data. This technology has significant application value in scenarios such as autonomous driving, robot navigation, and 3D reconstruction.

[0003] Currently, the mainstream approach to 3D object detection involves voxelizing 3D point clouds and then serializing them. These voxels are then grouped into multiple subsequences, which are then fed into a self-attention model for feature extraction. The extracted point cloud features are then used for object detection. However, a significant issue with the self-attention model is its quadratic complexity and high computational cost. For scenarios with large input data volumes, such as point clouds, it's difficult to balance speed and accuracy.

[0004] To address the technical challenges of 3D object detection, researchers have proposed numerous solutions, such as replacing the self-attention model with a general linear complexity model. However, these approaches lack specific design choices and result in poor performance. Other approaches attempt to improve model performance by leveraging large datasets, but this approach fails to address the inherently high computational complexity of the self-attention model, leading to lengthy training times and minimal accuracy gains. Summary of the Invention

[0005] In view of this, the present application provides a three-dimensional target detection model training method, a three-dimensional target detection method, a three-dimensional target detection model training device, a three-dimensional target detection device, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] One aspect of the present application provides a method for training a three-dimensional object detection model, comprising:

[0007] Obtaining a training data set, wherein the training data set includes a plurality of point cloud training data containing at least one detection target and target bounding box attribute information corresponding to each of the point cloud training data, wherein the target bounding box attribute information includes geometric parameters of a plurality of real bounding boxes in which the detection target is located and a type parameter of the detection target;

[0008] For each of the above point cloud training data, voxel processing is performed on the above point cloud training data based on a preset three-dimensional voxelization rule to obtain multiple grid point cloud features;

[0009] Process multiple above-mentioned grid point cloud features using a first initial encoding network to obtain target point cloud features, where the above-mentioned first initial encoding network is constructed based on a serialization module and a target linear complexity model;

[0010] Perform target detection processing on the above-mentioned target point cloud features using a second initial encoding network to obtain a target prediction result, where the initial detection model includes the above-mentioned first initial encoding network and the above-mentioned second initial encoding network;

[0011] Match the above-mentioned target prediction result with the target bounding box attribute information corresponding to the above-mentioned point cloud training data to obtain a target matching result;

[0012] Input the above-mentioned target matching result into a target loss function to obtain a target loss value, and iteratively adjust the network parameters of the above-mentioned initial detection model according to the above-mentioned target loss value, so as to obtain the above-mentioned three-dimensional target detection model.

[0013] According to an embodiment of the present application, the above-mentioned point cloud training data includes multiple point cloud information.

[0014] According to an embodiment of the present application, based on a preset three-dimensional voxelization rule, perform voxelization processing on the above-mentioned point cloud training data to obtain multiple grid point cloud features, including:

[0015] Calculate each above-mentioned point cloud information based on a voxel position calculation formula and map it on an initial three-dimensional grid map to obtain a target three-dimensional grid map, where the above-mentioned target three-dimensional grid map includes multiple voxel grids;

[0016] For each above-mentioned voxel grid, perform average pooling processing on multiple above-mentioned point cloud information within the voxel grid to obtain one above-mentioned grid point cloud feature.

[0017] According to an embodiment of the present application, the above-mentioned first initial encoding network includes m data processing structures, and the data processing structure includes the above-mentioned serialization module and the above-mentioned target linear complexity model.

[0018] According to an embodiment of the present application, using a first initial encoding network to process multiple above-mentioned grid point cloud features to obtain target point cloud features, including:

[0019] For the i-th data processing structure, perform downsampling processing on the input data to obtain sampled data, where the above-mentioned input data includes the above-mentioned grid point cloud features when i = 1 or the (i - 1)-th target sampling feature when 1 < i < m;

[0020] Process the above-mentioned sampled data using the above-mentioned serialization module to obtain a voxel sequence;

[0021] The voxel sequence is processed using the target linear complexity model to obtain an i-th target sampling feature, wherein when i=m, the i-th target sampling feature represents the target point cloud feature.

[0022] According to an embodiment of the present application, the above-mentioned serialization module is used to process the above-mentioned sampling data to obtain a voxel sequence, including:

[0023] Performing binary conversion on the sampled data to obtain formatted data, wherein the formatted data includes binary bits;

[0024] Based on a preset adjustment rule, bit adjustment processing is performed on the binary bits in the above format data to obtain adjusted data;

[0025] Performing Gray code decoding on the adjusted data to obtain decoded data;

[0026] Performing decimal conversion on the decoded data to obtain voxel position data corresponding to each voxel grid;

[0027] The plurality of voxel position data are processed using a Hilbert curve to obtain the voxel sequence.

[0028] According to an embodiment of the present application, the target linear complexity model is used to process the voxel sequence to obtain the i-th target sampling feature, including:

[0029] Processing the voxel sequence using a state-space model to obtain a model weight of the state-space model;

[0030] generating an output vector according to a state vector, the voxel sequence, and a parameter matrix set, wherein the state vector is determined according to the model weights;

[0031] The above output vector is upsampled to obtain the above i-th target sampling feature.

[0032] According to an embodiment of the present application, target detection processing is performed on the target point cloud features using the second initial encoding network to obtain a target prediction result, including:

[0033] Downsampling the target point cloud features using a convolutional structure to obtain target convolutional features, wherein the convolutional structure includes a plurality of three-dimensional convolutional layers connected in sequence;

[0034] The target convolution features are classified and predicted using a classification regression structure to obtain a predicted category vector for each of the detected targets and a predicted bounding box attribute vector for the detected target, wherein the target prediction result includes the predicted category vector and the predicted bounding box attribute vector.

[0035] According to an embodiment of the present application, the target prediction result includes a plurality of predicted category vectors and predicted bounding box attribute vectors of the detected targets.

[0036] According to an embodiment of the present application, the target prediction result and the target bounding box attribute information corresponding to the point cloud training data are matched to obtain a target matching result, including:

[0037] For each of the detected targets, generating an overlapped matching matrix for the detected target based on the predicted bounding box attribute vector and the plurality of true bounding boxes in the target bounding box attribute information, wherein the overlapped matching matrix includes a plurality of matrix parameters;

[0038] For any of the above matrix parameters, when the above matrix parameters meet the preset threshold, the type parameters of the detection target corresponding to the above matrix parameters are determined as the target type, and the type score of the above detection target as the above target type is calculated, wherein the above target matching result includes the above overlapping matching matrix and the above target type.

[0039] According to an embodiment of the present application, the target matching result is input into the target loss function to obtain the target loss value, including:

[0040] Calculate a classification loss value based on the multiple target types and the type score of each target type;

[0041] Calculate the regression loss value based on the above overlapping matching matrix;

[0042] According to the above classification loss value and the above regression loss value, the above target loss value is generated.

[0043] Another aspect of the present application provides a three-dimensional object detection method, comprising:

[0044] Acquire point cloud data to be processed containing at least one detection object;

[0045] The point cloud data to be processed is input into a three-dimensional object detection model, and an object detection result is output, wherein the object detection result includes the object type of each of the detected objects and the geometric parameters of the detection bounding box of the detected object.

[0046] Another aspect of the present application provides a training device for a three-dimensional object detection model, comprising:

[0047] a first acquisition module, configured to acquire a training data set, wherein the training data set includes a plurality of point cloud training data containing at least one detection target and target bounding box attribute information corresponding to each of the point cloud training data, wherein the target bounding box attribute information includes geometric parameters of a plurality of real bounding boxes in which the detection target is located and a type parameter of the detection target;

[0048] A voxelization module is used to voxelize each of the above-mentioned point cloud training data based on a preset three-dimensional voxelization rule to obtain multiple grid point cloud features;

[0049] A first encoding module is configured to process the plurality of grid point cloud features using a first initial encoding network to obtain a target point cloud feature, wherein the first initial encoding network is constructed based on a serialization module and a target linear complexity model;

[0050] A second encoding module is used to perform target detection processing on the target point cloud features using a second initial encoding network to obtain a target prediction result, wherein the initial detection model includes the first initial encoding network and the second initial encoding network;

[0051] A matching module is used to match the target prediction result with the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result;

[0052] The training module is used to input the above target matching results into the target loss function to obtain the target loss value, so as to iteratively adjust the network parameters of the above initial detection model according to the above target loss value, thereby obtaining the above three-dimensional target detection model.

[0053] Another aspect of the present application provides a three-dimensional object detection device, comprising:

[0054] A second acquisition module is used to acquire the to-be-processed point cloud data containing at least one detection object;

[0055] The detection module is used to input the above-mentioned point cloud data to be processed into the three-dimensional target detection model and output the target detection results, wherein the above-mentioned target detection results include the object type of each of the above-mentioned detected objects and the geometric parameters of the detection bounding box of the above-mentioned detected objects.

[0056] Another aspect of the present application provides an electronic device, comprising:

[0057] one or more processors;

[0058] a memory for storing one or more programs,

[0059] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0060] Another aspect of the present application provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0061] Another aspect of the present application provides a computer program product, which includes computer-executable instructions. When the instructions are executed, the instructions are used to implement the method described above.

[0062] According to an embodiment of the present application, point cloud training data in a training data set is voxelized based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features; a first initial coding network constructed based on a serialization module and a target linear complexity model is used to process the plurality of grid point cloud features to obtain target point cloud features; a second initial coding network is used to perform target detection processing on the target point cloud features to obtain a target prediction result; the target prediction result and the target bounding box attribute information corresponding to the point cloud training data are matched to obtain a target matching result; the target matching result is input into a target loss function to obtain a target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model. Since the first initial coding network in the target detection model of the present application is constructed based on a serialization module and a target linear complexity model, the computational cost of the point cloud features can be reduced, thereby improving the performance of the three-dimensional target detection model, and further improving the speed and accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The above and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0064] Figure 1 A flowchart of a method for training a three-dimensional object detection model according to an embodiment of the present application is shown;

[0065] Figure 2 A schematic diagram of forward propagation training of a target linear complexity model according to an embodiment of the present application is shown;

[0066] Figure 3 A schematic diagram of back propagation training of a target linear complexity model according to an embodiment of the present application is shown;

[0067] Figure 4 A schematic diagram of the network structure of a three-dimensional object detection model according to an embodiment of the present application is shown;

[0068] Figure 5A flowchart of a three-dimensional object detection method according to an embodiment of the present application is shown;

[0069] Figure 6 A block diagram of a training device for a three-dimensional object detection model according to an embodiment of the present application is shown;

[0070] Figure 7 A block diagram of a three-dimensional object detection device according to an embodiment of the present application is shown;

[0071] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0072] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0073] The terms used herein are only for describing specific embodiments and are not intended to limit this application. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0074] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0075] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0076] In the embodiments of this application, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of all data involved (including, but not limited to, user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and maintain the security of user personal information and network security.

[0077] In the embodiments of the present application, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0078] Figure 1 A flowchart of a method for training a three-dimensional object detection model according to an embodiment of the present application is shown.

[0079] like Figure 1 As shown, the training method of the three-dimensional target detection model includes operations S101 to S103.

[0080] In operation S101, a training data set is obtained, where the training data set includes a plurality of point cloud training data including at least one detection target and target bounding box attribute information corresponding to each point cloud training data, where the target bounding box attribute information includes geometric parameters of a plurality of real bounding boxes in which the detection target is located and type parameters of the detection target;

[0081] In operation S102, for each point cloud training data, voxel processing is performed on the point cloud training data based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features;

[0082] In operation S103, a first initial encoding network is used to process the plurality of grid point cloud features to obtain a target point cloud feature, wherein the first initial encoding network is constructed based on a serialization module and a target linear complexity model;

[0083] In operation S104, target detection processing is performed on the target point cloud features using the second initial encoding network to obtain a target prediction result, wherein the initial detection model includes the first initial encoding network and the second initial encoding network;

[0084] In operation S105, matching processing is performed on the target prediction result and the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result;

[0085] In operation S106 , the target matching result is input into the target loss function to obtain a target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model.

[0086] According to an embodiment of the present application, the point cloud training data contains the coordinate information of all points. However, because the point cloud data is collected by radar, the point cloud training data also includes an additional feature attribute, namely the reflectance value. The geometric parameters can refer to the length, width, height and center coordinates of the three-dimensional real-world bounding box.

[0087] According to an embodiment of the present application, the type parameter of the detection target may indicate the type of the detection target (eg, a specific species or model of a cat, dog, or plant).

[0088] According to an embodiment of the present application, point cloud voxelization is performed on each point cloud training sample to obtain a regular three-dimensional grid structure. Point cloud voxelization divides the three-dimensional space into a fixed-size cubic grid. Each point is mapped to a corresponding voxel unit according to its coordinates. Average pooling is used in the voxel unit to process point cloud features, simplifying the data while retaining important geometric features. Multiple grid point cloud features can be obtained through voxelization, each of which represents the overall characteristics of a cubic grid.

[0089] According to an embodiment of the present application, the grid point cloud features obtained after voxelization are input into the first initial encoding network to be processed by the serialization module and the target linear complexity model therein to obtain the target point cloud features Then the target point cloud features The target prediction result is input into the second initial encoding network for target detection processing, and the target prediction result characterizes the target type of each detected target and the boundary parameters of the predicted bounding box where the detected target is located.

[0090] According to an embodiment of the present application, the predicted target prediction result is matched with the target bounding box attribute information to obtain a target matching result, which is then input into a target loss function, thereby iteratively adjusting the network parameters of the initial detection model based on the obtained target loss value, thereby obtaining a three-dimensional target detection model.

[0091] According to an embodiment of the present application, point cloud training data in a training data set is voxelized based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features; a first initial coding network constructed based on a serialization module and a target linear complexity model is used to process the plurality of grid point cloud features to obtain target point cloud features; a second initial coding network is used to perform target detection processing on the target point cloud features to obtain a target prediction result; the target prediction result and the target bounding box attribute information corresponding to the point cloud training data are matched to obtain a target matching result; the target matching result is input into a target loss function to obtain a target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model. Since the first initial coding network in the target detection model of the present application is constructed based on a serialization module and a target linear complexity model, the computational cost of the point cloud features can be reduced, thereby improving the performance of the three-dimensional target detection model, and further improving the speed and accuracy of target detection.

[0092] According to an embodiment of the present application, the point cloud training data includes a plurality of point cloud information.

[0093] According to an embodiment of the present application, based on a preset three-dimensional voxelization rule, voxelization is performed on the point cloud training data to obtain multiple grid point cloud features, including:

[0094] Calculating each point cloud information based on a voxel position calculation formula to map it onto an initial three-dimensional grid map to obtain a target three-dimensional grid map, wherein the target three-dimensional grid map includes a plurality of voxel grids;

[0095] For each voxel grid, the multiple point cloud information within the voxel grid is averaged and pooled to obtain a grid point cloud feature.

[0096] According to the embodiment of the present application, it is assumed that there is a point cloud training data in a three-dimensional space Each of these Represents a point in the point cloud, with three-dimensional coordinates , is the reflection value. The goal of the voxelization process is to map these irregularly distributed points onto an initial three-dimensional grid map. The initial three-dimensional grid map consists of a size of Define the voxel coordinates in a voxel grid in the initial 3D grid map , the position of each voxel can be calculated by formula (1)

[0097] (1)

[0098] in Indicates a round-down operation. is the side length of the voxel. This formula represents each point in the point cloud corresponding to a voxel in the voxel grid .

[0099] According to an embodiment of the present application, due to the characteristics of the voxelization operation, each point can only be mapped to a unique voxel grid, and a voxel grid can contain multiple points. Therefore, assuming that there are M points in the j-th voxel grid after voxelization, the set of all point cloud features is , where , is the three-dimensional coordinate, is the reflection value. Then the process of average pooling can be described by formula (2):

[0100] (2)

[0101] According to an embodiment of the present application, through average pooling, a voxel grid containing points can be simplified to only one point, and the feature of this one point is used to replace the overall feature of the th voxel grid, that is, the grid point cloud feature.

[0102] According to an embodiment of the present application, the first initial coding network includes m data processing structures, and the data processing structure includes a serialization module and a target linear complexity model.

[0103] According to an embodiment of the present application, using the first initial coding network to process multiple grid point cloud features to obtain target point cloud features includes:

[0104] For the i-th data processing structure, perform downsampling on the input data to obtain sampled data, where the input data includes the grid point cloud feature when i = 1 or the (i - 1)-th target sampling feature when 1 < i < m;

[0105] Use the serialization module to process the sampled data to obtain a voxel sequence;

[0106] Use the target linear complexity model to process the voxel sequence to obtain the i-th target sampling feature, where when i = m, the i-th target sampling feature represents the target point cloud feature.

[0107] According to an embodiment of the present application, input the voxelized grid point cloud feature into the first initial coding network, and output the target point cloud feature after feature extraction. Among them, the dimension of the output target point cloud feature is the same as the dimension of the input voxelized grid point cloud feature. That is, for the above-mentioned voxelized grid point cloud feature (where is the number of voxel grids), which is input into the first initial encoding network, and is processed by downsampling, serialization module and target linear complexity model in sequence to obtain the extracted target point cloud features Among them, the first initial encoding network is constructed by the serialization module and the target linear complexity model, and m layers are stacked.

[0108] According to an embodiment of the present application, the sampling data is processed using a serialization module to obtain a voxel sequence, including:

[0109] Performing binary conversion on the sampled data to obtain format data, wherein the format data includes binary bits;

[0110] Based on a preset adjustment rule, bit adjustment processing is performed on the binary bits in the format data to obtain adjusted data;

[0111] Performing Gray code decoding on the adjusted data to obtain decoded data;

[0112] Perform decimal conversion on the decoded data to obtain the voxel position data corresponding to each voxel grid;

[0113] The Hilbert curve is used to process multiple voxel position data to obtain a voxel sequence.

[0114] According to the embodiment of the present application, in order to improve the proximity of voxels in serialization, the serialization module uses the characteristics of space filling curves that can traverse all elements in the space without duplication and retain the spatial topology for serialization, mainly using the Hilbert space filling curve. The coordinate representation of the voxel feature is The goal of the serialization module is to map voxels to their positions within the Hilbert curve. superior.

[0115] Specifically, first sample the data Convert to have Bit binary format, get format data, such as coordinates is converted to ,in Then process the format data from high to low The bit value of the bit is swapped and reversed to adjust the order of the bits: swap the bit order when the current bit is 0; otherwise, reverse it. The bit values ​​are concatenated into a long bit string and apply it globally The Gray code is decoded by multiple times to obtain a binary string of decoded data, which is converted into a decimal number to obtain the voxel position data corresponding to each voxel grid. Therefore, the coordinates of each voxel can be calculated as a position on the Hilbert curve , according to the calculated position , all voxels can be sorted into a single sequence while maintaining their spatial proximity, which is the voxel sequence.

[0116] Figure 2 A schematic diagram of forward propagation training of a target linear complexity model according to an embodiment of the present application is shown. Figure 3 A schematic diagram of back-propagation training of a target linear complexity model according to an embodiment of the present application is shown.

[0117] According to an embodiment of the present application, a target linear complexity model is used to process a voxel sequence to obtain an i-th target sampling feature, including:

[0118] Processing the voxel sequence using the state-space model to obtain the model weight of the state-space model;

[0119] generating an output vector according to a state vector, a voxel sequence, and a parameter matrix set, wherein the state vector is determined according to the model weights;

[0120] The output vector is upsampled to obtain the i-th target sampling feature.

[0121] According to an embodiment of the present application, the target linear complexity model includes a feature extraction layer based on a state-space model. State-space models (SSMs) provide a robust framework for modeling physical systems, particularly for linear time-invariant (LTI) systems. These models model the dynamics of the system state via a set of first-order equations, defined as follows:

[0122]

[0123] (3)

[0124] in, represents the state vector, matrix and Used to describe status ,enter and output The relationship between is the voxel sequence, is the output vector, and the two formulas in formula (3) are called the update rule and the output rule respectively.

[0125] Since SSM operates in a continuous sequence Therefore, it cannot directly process discrete inputs such as discrete features of point clouds. Therefore, the SSM needs to be converted into a discrete version, using the zero-order hold technique to discretize the matrix ,in is the discrete time step, is the exponential mapping of the matrix. The discretized SSM can be expressed by formula (4):

[0126]

[0127] (4)

[0128] In this discrete form, the state It is associated with the state of the previous moment through a recurrence relationship. is the parameter matrix. Typically used as a residual connection, it may be simplified or ignored in some cases. The state-space model takes the input labeled Mapping to output tags On the one hand, this operation is efficient because the update rule and the output rule take a constant time for each point cloud data, and the context information is compressed to a fixed size, that is, the hidden state On the other hand, its performance in long contexts is affected by its hidden state The limitation of representation ability. Therefore, the key is to model a hidden state with strong representation ability. This application draws on the experience of large models in the field of compressing a lot of context into model weights to convert hidden states into Implemented as a deep learning model consisting of linear layers The model weight , which aims to compress the rich prior knowledge in large amounts of unlabeled data into deep learning models Weight , then the above state space model can be expressed by formula (5):

[0129]

[0130] (5)

[0131] in For this deep learning model The learning rate during training is set to . To update the model weights Specifically, the loss function is defined by constructing a self-supervised regression task using a large amount of unlabeled data: for the input point cloud , deep learning model Need to discover The correlation between dimensions in order to reconstruct it from partial information and update its weights in the process . Using two learnable matrices Process the original input point cloud (i.e., voxel sequence) separately and generate the damaged training views separately and tab view The loss function is defined as formula (6):

[0132] (6)

[0133] The model forward reasoning process is defined as: ,in It is also a learnable matrix used to adjust the dimension. It is called the test view.

[0134] Therefore, the first initial encoding network will update the deep learning model through the gradient of the above loss function during forward propagation The model weight ,at this time Keep it fixed and can be regarded as a hyperparameter, see Figure 2 . During back propagation, Update together with other parameters in the second initial encoding network, see Figure 3 In general, the first initial encoding network is constructed by a serialization module and a specially designed target linear complexity model, and multiple layers are stacked. In each layer, the point cloud is first downsampled, then input into the serialization module and the specially designed target linear complexity model, and finally restored to the original resolution by upsampling. The first initial encoding network extracts the voxelized grid point cloud features. The features of the target point cloud are obtained .

[0135] It should be noted that Figure 2 and Figure 3 The blue parts in the graph represent fixed parameters.

[0136] Figure 4 A schematic diagram of the network structure of a three-dimensional target detection model according to an embodiment of the present application is shown.

[0137] According to the embodiments of this application, see Figure 4 , use the second initial encoding network to perform target detection processing on the target point cloud features to obtain the target prediction results, including:

[0138] The target point cloud features are downsampled using a convolutional structure to obtain target convolutional features, wherein the convolutional structure includes multiple three-dimensional convolutional layers connected in sequence;

[0139] The target convolution features are classified and predicted using a classification regression structure to obtain a predicted category vector for each detected target and a predicted bounding box attribute vector where the detected target is located. The target prediction result includes a predicted category vector and a predicted bounding box attribute vector.

[0140] According to an embodiment of the present application, the target point cloud features after the first initial coding network extraction are The input is sent to the second initial coding network for target detection to obtain the target prediction result. The second initial coding network includes a convolutional structure and a classification and regression structure, and the classification and regression structure includes a classification branch and a regression branch.

[0141] According to the embodiment of the present application, the convolution structure can use 4 or other number of convolution layers to further process the feature map and further extract high-level features related to the detection task. Take 4 convolution layers as an example: for the target point cloud features after the first initial encoding network feature extraction , down-sampling is performed through 4 convolutional layers, and the process is respectively the 4 formulas in formula (7):

[0142]

[0143]

[0144]

[0145] (7)

[0146] in represents a three-dimensional convolution operation, is the convolution kernel size, is the convolution step size, The parameter determines the size of the output data in the feature dimension after convolution. Applied after the convolution operation, it normalizes the output of the convolution. Specifically, it normalizes the data by calculating the mean and variance within the batch. It is a commonly used activation function used to introduce nonlinearity. It maps negative values ​​to zero and leaves positive values ​​unchanged.

[0147] According to the embodiment of the present application, after downsampling of the convolution structure, the target convolution feature is finally output ,in is the number of predicted bounding boxes, at which point the features have been highly extracted and compressed.

[0148] According to an embodiment of the present application, the classification branch is adjusted by the convolution layer dimension, the output vector is the predicted category vector , represents the category score of the predicted target, where is the number of categories, Any element Representative The predicted bounding box for The score of the category is calculated as shown in formula (8):

[0149] (8)

[0150] in, , is the i-th element of the input vector z, N is the dimension of the input vector, and e is the base of the natural logarithm.

[0151] According to an embodiment of the present application, the regression branch formula is adjusted by the convolution layer dimension, the output vector is the predicted bounding box attribute vector , used to regress the three-dimensional bounding box of the target. Among them, 6 represents the center coordinates of the prediction box and the length, width and height of the prediction box, that is, , the specific calculation is shown in formula (9):

[0152] (9)

[0153] According to an embodiment of the present application, the target prediction result includes a predicted category vector and a predicted bounding box attribute vector of multiple detected targets.

[0154] According to an embodiment of the present application, matching processing is performed on the target prediction result and the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result, including:

[0155] For each detected target, generating an overlapping matching matrix of the detected target based on the predicted bounding box attribute vector and multiple real bounding boxes in the target bounding box attribute information, wherein the overlapping matching matrix includes multiple matrix parameters;

[0156] For any matrix parameter, when the matrix parameter meets the preset threshold, the type parameter of the detection target corresponding to the matrix parameter is determined as the target type, and the type score of the detection target as the target type is calculated, where the target matching result includes the overlapping matching matrix and the target type.

[0157] According to the embodiment of the present application, the target prediction result and target bounding box attribute information in the training data set Matching is performed, and the matched target matching result is input into the loss function to obtain a target loss value; the network parameters of the initial detection model are iteratively adjusted according to the target loss value to obtain a trained 3D target detection model. The target detection model includes a first initial encoding network and a second initial encoding network.

[0158] According to the embodiment of the present application, the specific matching process is: Initialize the matrix with all zeros , for the Predicted bounding boxes, calculate their differences from all true bounding boxes of Value, find the type score of the predicted bounding box The largest ground-truth bounding box (where represents the number of real labeled objects in the dataset). The calculation method of is shown in formula (10):

[0159] (10)

[0160] The intersection is the volume of the area where the predicted bounding box and the true bounding box overlap. The union is the total volume of the area contained by the predicted bounding box and the true box. (i.e., matrix parameters and type scores) exceeds the preset threshold of 0.7 (i.e., the preset threshold, other values ​​can be set as needed), then the true category of the predicted bounding box is set to the target type of the true bounding box ,Right now Is set to 1, representing the The predicted bounding box corresponds to the category , if the maximum If the predicted bounding box is smaller than the threshold, it is considered as the background class. At the same time, the above matching process will also generate a matching matrix consisting of 0 and 1 , that is, the overlapping matching matrix, which represents predicted bounding boxes and The matching relationship between the real object boxes of the dataset.

[0161] According to an embodiment of the present application, the target matching result is input into the target loss function to obtain the target loss value, including:

[0162] Calculate the classification loss value based on multiple target types and the type score of each target type;

[0163] Calculate the regression loss value based on the overlap matching matrix;

[0164] Generate the target loss value based on the classification loss value and regression loss value.

[0165] According to an embodiment of the present application, the target loss function consists of two parts: a branch classification loss function and a regression branch loss function. The process of calculating the classification loss value based on the branch classification loss function is shown in formula (11), and the process of calculating the regression loss value based on the regression branch loss function is shown in formula (12):

[0166] (11)

[0167] in: , is through The obtained category probability, It is The predicted bounding box for The score of the target category, i.e. The elements in the vector. Is the true category label in one-hot encoding form, representing the Whether the predicted bounding box corresponds to the target category .

[0168] (12)

[0169] in is the regression parameter of the predicted bounding box, i.e., the geometric parameter, represented by the vector get. , are the geometric parameters of the ground-truth bounding box, obtained from the dataset.

[0170]

[0171] Target loss value ,in is a hyperparameter used to adjust the balance and is set to .

[0172] According to the embodiment of the present application, the overall target loss value is obtained Then, the network parameters of the initial detection model (including , but not including ), and obtain the trained 3D object detection model.

[0173] According to the embodiments of the present application, the computational cost of point cloud feature extraction is significantly reduced by replacing the self-attention model in the prior art with a specially designed target linear complexity model. The complexity of the self-attention model is quadratic complexity. ,in is the size of the input data, and the target linear complexity model reduces this complexity to This design significantly reduces computational time when processing large-scale point cloud data. For example, experimental data shows that on typical point cloud datasets (such as the KITTI dataset), the proposed model reduces inference time by approximately 40% and training time by over 30%, while maintaining similar accuracy to the self-attention model.

[0174] According to the embodiments of this application, by defining hidden states in state space as weights of a deep learning model (i.e., a state-space model) and constructing a self-supervised task using a large amount of unlabeled data to update the model, the present application improves the representation of hidden states, further enhances the feature extraction capabilities of the first initial encoding network, and improves the accuracy of the model. Specific experimental results show that when processing large-scale point cloud data (e.g., millions of points), the present application improves accuracy indicators such as mean average precision (mAP) by approximately 5%, while maintaining low computing resource consumption.

[0175] In summary, this application effectively solves the problem of high computational cost of the self-attention model in the existing technology through an innovative target linear complexity model, and achieves a balance between accuracy and speed. It is particularly suitable for application scenarios such as autonomous driving and robot navigation that require real-time and high precision.

[0176] Figure 5 A flowchart of a three-dimensional target detection method according to an embodiment of the present application is shown.

[0177] like Figure 5 As shown, the three-dimensional target detection method includes operations S501-S502.

[0178] In operation S501, to-be-processed point cloud data including at least one detection object is acquired;

[0179] In operation S502 , the point cloud data to be processed is input into a three-dimensional object detection model, and an object detection result is output, wherein the object detection result includes an object type of each detected object and geometric parameters of a detection bounding box of the detected object.

[0180] According to an embodiment of the present application, the detection object can be a person or any type of animal or plant. The point cloud data to be processed can be collected by radar. The collected point cloud data to be processed is input into a trained three-dimensional target detection model to obtain the object type of each detection object in the point cloud data to be processed and the detection bounding box of the detection object, including the length, width, height and center point coordinates of the detection bounding box.

[0181] According to an embodiment of the present application, the point cloud data to be processed is input into a three-dimensional target detection model, and the target detection result is output. Since the first initial encoding network in the three-dimensional target detection model is constructed based on the serialization module and the target linear complexity model, the computational cost of the point cloud features can be reduced, thereby improving the detection speed and accuracy of the three-dimensional target.

[0182] Figure 6 A block diagram of a training device for a three-dimensional object detection model according to an embodiment of the present application is shown.

[0183] like Figure 6 As shown, the training device 600 for the three-dimensional target detection model includes a first acquisition module 610, a voxelization module 620, a first encoding module 630, a second encoding module 640, a matching module 650, and a training module 660.

[0184] A first acquisition module 610 is configured to acquire a training data set, wherein the training data set includes a plurality of point cloud training data containing at least one detection target and target bounding box attribute information corresponding to each point cloud training data, wherein the target bounding box attribute information includes geometric parameters of a plurality of real bounding boxes in which the detection target is located and type parameters of the detection target;

[0185] A voxelization module 620 is configured to voxelize each point cloud training data based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features;

[0186] A first encoding module 630 is configured to process the plurality of grid point cloud features using a first initial encoding network to obtain target point cloud features, wherein the first initial encoding network is constructed based on the serialization module and the target linear complexity model;

[0187] A second encoding module 640 is configured to perform target detection processing on the target point cloud features using a second initial encoding network to obtain a target prediction result, wherein the initial detection model includes the first initial encoding network and the second initial encoding network;

[0188] Matching module 650, used to match the target prediction result with the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result;

[0189] The training module 660 is used to input the target matching result into the target loss function to obtain the target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model.

[0190] According to an embodiment of the present application, point cloud training data in a training data set is voxelized based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features; a first initial coding network constructed based on a serialization module and a target linear complexity model is used to process the plurality of grid point cloud features to obtain target point cloud features; a second initial coding network is used to perform target detection processing on the target point cloud features to obtain a target prediction result; the target prediction result and the target bounding box attribute information corresponding to the point cloud training data are matched to obtain a target matching result; the target matching result is input into a target loss function to obtain a target loss value, so as to iteratively adjust the network parameters of the initial detection model according to the target loss value, thereby obtaining a three-dimensional target detection model. Since the first initial coding network in the target detection model of the present application is constructed based on a serialization module and a target linear complexity model, the computational cost of the point cloud features can be reduced, thereby improving the performance of the three-dimensional target detection model, and further improving the speed and accuracy of target detection.

[0191] Figure 7 A block diagram of a three-dimensional object detection device according to an embodiment of the present application is shown.

[0192] like Figure 7 As shown, the three-dimensional object detection device 700 includes a second acquisition module 710 and a detection module 720.

[0193] The second acquisition module 710 is used to acquire the to-be-processed point cloud data containing at least one detection object;

[0194] The detection module 720 is used to input the point cloud data to be processed into the three-dimensional object detection model and output the object detection results, wherein the object detection results include the object type of each detected object and the geometric parameters of the detection bounding box of the detected object.

[0195] According to an embodiment of the present application, the point cloud data to be processed is input into a three-dimensional target detection model, and the target detection result is output. Since the first initial encoding network in the three-dimensional target detection model is constructed based on the serialization module and the target linear complexity model, the computational cost of the point cloud features can be reduced, thereby improving the detection speed and accuracy of the three-dimensional target.

[0196] According to the embodiments of the present application, any number of modules, submodules, units, and subunits, or at least part of the functions of any number of them, can be implemented in one module. According to the embodiments of the present application, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present application, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present application, one or more of the modules, submodules, units, and subunits can be at least partially implemented as a computer program module, which can perform the corresponding functions when the computer program module is executed.

[0197] For example, any of the first acquisition module 610, the voxelization module 620, the first encoding module 630, the second encoding module 640, the matching module 650, the training module 660, or the second acquisition module 710 and the detection module 720 can be combined into a single module / unit / sub-unit for implementation, or any of these modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functionality of one or more of these modules / units / sub-units can be combined with at least part of the functionality of other modules / units / sub-units and implemented in a single module / unit / sub-unit. According to an embodiment of the present application, at least one of the first acquisition module 610, the voxelization module 620, the first encoding module 630, the second encoding module 640, the matching module 650, the training module 660, or the second acquisition module 710 and the detection module 720 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of circuit integration or packaging, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first acquisition module 610, the voxelization module 620, the first encoding module 630, the second encoding module 640, the matching module 650, the training module 660, or the second acquisition module 710 and the detection module 720 may be at least partially implemented as a computer program module, which, when executed, may perform the corresponding function.

[0198] It should be noted that the training device of the three-dimensional target detection model or the three-dimensional target detection device part in the embodiments of the present application corresponds to the training method of the three-dimensional target detection model or the three-dimensional target detection method part in the embodiments of the present application. The description of the training device of the three-dimensional target detection model or the three-dimensional target detection device part specifically refers to the training method of the three-dimensional target detection model or the three-dimensional target detection method part, and will not be repeated here.

[0199] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present application is shown. Figure 8 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0200] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present application includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage unit 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present application.

[0201] Various programs and data required for the operation of the electronic device 800 are stored in the RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in the one or more memories.

[0202] According to an embodiment of the present application, electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to bus 804. Electronic device 800 may also include one or more of the following components connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. Communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 810 as needed, so that computer programs read from the removable media can be installed into storage section 808 as needed.

[0203] According to an embodiment of the present application, the method flow according to the embodiment of the present application can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present application are executed. According to an embodiment of the present application, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0204] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of this application is implemented.

[0205] According to embodiments of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0206] For example, according to an embodiment of the present application, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than the ROM 802 and RAM 803 .

[0207] An embodiment of the present application also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present application. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiment of the present application.

[0208] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present application are performed. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0209] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0210] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0211] The embodiments of the present application have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The present application does not depart from the scope of the present application, and those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present application.

Claims

1. A training method for a three-dimensional object detection model, characterized in that: Including: Obtain a training data set, where the training data set includes multiple point cloud training data containing at least one detection target and target bounding box attribute information corresponding to each point cloud training data, and the target bounding box attribute information includes geometric parameters of multiple true bounding boxes where the detection target is located and type parameters of the detection target; For each point cloud training data, based on a preset three-dimensional voxelization rule, perform voxelization processing on the point cloud training data to obtain multiple grid point cloud features; Use a first initial encoding network to process the multiple grid point cloud features to obtain target point cloud features, where the first initial encoding network is constructed based on a serialization module and a target linear complexity model; Use a second initial encoding network to perform target detection processing on the target point cloud features to obtain a target prediction result, where the initial detection model includes the first initial encoding network and the second initial encoding network; Perform matching processing on the target prediction result and the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result; Input the target matching result into a target loss function to obtain a target loss value, and iteratively adjust the network parameters of the initial detection model according to the target loss value, so as to obtain the three-dimensional target detection model; Where the first initial encoding network includes m data processing structures, and the data processing structure includes the serialization module and the target linear complexity model; Where using the first initial encoding network to process the multiple grid point cloud features to obtain target point cloud features includes: For the i-th data processing structure, perform downsampling processing on the input data to obtain sampled data, where the input data includes the grid point cloud features when i = 1 or the (i - 1)-th target sampled features when 1 < i < m; Use the serialization module to process the sampled data to obtain a voxel sequence; Use the target linear complexity model to process the voxel sequence to obtain the i-th target sampled feature, where when i = m, the i-th target sampled feature represents the target point cloud features; Where using the target linear complexity model to process the voxel sequence to obtain the i-th target sampled feature includes: Use a state space model to process the voxel sequence to obtain the model weights of the state space model; Generate an output vector according to a state vector, the voxel sequence, and a set of parameter matrices, where the state vector is determined according to the model weights; Perform upsampling processing on the output vector to obtain the i-th target sampled feature.

2. The method according to claim 1, characterized in that The point cloud training data includes multiple point cloud information; Where based on a preset three-dimensional voxelization rule, performing voxelization processing on the point cloud training data to obtain multiple grid point cloud features includes: ​ For each voxel grid, average pooling processing is performed on the multiple point cloud information in the voxel grid to obtain a grid point cloud feature.

3. The method according to claim 1, characterized in that Processing the sampled data using the serialization module to obtain a voxel sequence includes: Performing binary conversion on the sampled data to obtain format data, wherein the format data includes binary bits; Based on a preset adjustment rule, performing bit adjustment processing on the binary bits in the formatted data to obtain adjusted data; performing Gray code decoding processing on the adjusted data to obtain decoded data; Performing decimal conversion on the decoded data to obtain voxel position data corresponding to each voxel grid; The plurality of voxel position data are processed using a Hilbert curve to obtain the voxel sequence.

4. The method according to claim 1, wherein Utilizing the second initial coding network to perform target detection processing on the target point cloud features to obtain a target prediction result, including: Downsampling the target point cloud features using a convolutional structure to obtain target convolutional features, wherein the convolutional structure includes a plurality of three-dimensional convolutional layers connected in sequence; The target convolution features are classified and predicted using a classification regression structure to obtain a predicted category vector for each of the detected targets and a predicted bounding box attribute vector of the detected target, wherein the target prediction result includes the predicted category vector and the predicted bounding box attribute vector.

5. The method according to claim 1, wherein The target prediction result includes a plurality of predicted category vectors and predicted bounding box attribute vectors of the detected targets; The target prediction result and the target bounding box attribute information corresponding to the point cloud training data are matched to obtain a target matching result, including: For each of the detected targets, generating an overlapped matching matrix of the detected target based on the predicted bounding box attribute vector and a plurality of the real bounding boxes in the target bounding box attribute information, wherein the overlapped matching matrix includes a plurality of matrix parameters; For any of the matrix parameters, if the matrix parameters meet a preset threshold, determining the type parameter of the detection target corresponding to the matrix parameter as the target type, and calculating the type score of the detection target being the target type, wherein the target matching result includes the overlapping matching matrix and the target type; The target matching result is input into the target loss function to obtain the target loss value, including: Calculating a classification loss value according to the plurality of target types and a type score for each target type; Calculating a regression loss value according to the overlapping matching matrix; The target loss value is generated according to the classification loss value and the regression loss value.

6. A three-dimensional target detection method, characterized in that: include: Acquire point cloud data to be processed containing at least one detection object; Inputting the to-be-processed point cloud data into a three-dimensional object detection model and outputting an object detection result, wherein the object detection result includes an object type of each of the detected objects and geometric parameters of a detection bounding box of the detected object; The three-dimensional target detection model is trained by the method according to any one of claims 1 to 5.

7. A training device for a three-dimensional object detection model, characterized in that: include: A first acquisition module, configured to acquire a training data set, where the training data set includes a plurality of point cloud training data each containing at least one detection target and target bounding box attribute information corresponding to each point cloud training data, and the target bounding box attribute information includes geometric parameters of a plurality of true bounding boxes where the detection target is located and type parameters of the detection target; A voxelization module, configured to perform voxelization processing on each point cloud training data based on a preset three-dimensional voxelization rule to obtain a plurality of grid point cloud features; A first encoding module, configured to process the plurality of grid point cloud features by using a first initial encoding network to obtain target point cloud features, where the first initial encoding network is constructed based on a serialization module and a target linear complexity model; A second encoding module, configured to perform target detection processing on the target point cloud features by using a second initial encoding network to obtain a target prediction result, where the initial detection model includes the first initial encoding network and the second initial encoding network; A matching module, configured to perform matching processing on the target prediction result and the target bounding box attribute information corresponding to the point cloud training data to obtain a target matching result; A training module, configured to input the target matching result into a target loss function to obtain a target loss value, and iteratively adjust network parameters of the initial detection model according to the target loss value, so as to obtain the three-dimensional target detection model; Wherein, the first initial encoding network includes m data processing structures, and the data processing structure includes the serialization module and the target linear complexity model; Wherein, processing the plurality of grid point cloud features by using the first initial encoding network to obtain target point cloud features includes: For the i-th data processing structure, performing downsampling processing on input data to obtain sampled data, where the input data includes the grid point cloud features when i = 1 or the (i - 1)-th target sampling features when 1 < i < m; Processing the sampled data by using the serialization module to obtain a voxel sequence; Processing the voxel sequence by using the target linear complexity model to obtain the i-th target sampling feature, where when i = m, the i-th target sampling feature represents the target point cloud features; Wherein, processing the voxel sequence by using the target linear complexity model to obtain the i-th target sampling feature includes: Processing the voxel sequence by using a state space model to obtain model weights of the state space model; Generating an output vector according to a state vector, the voxel sequence, and a set of parameter matrices, where the state vector is determined according to the model weights; Performing upsampling processing on the output vector to obtain the i-th target sampling feature.

8. A three-dimensional target detection device, characterized in that: Including: A second acquisition module, configured to acquire point cloud data to be processed containing at least one detection object; A detection module, configured to input the point cloud data to be processed into the three-dimensional target detection model and output a target detection result, where the target detection result includes the object type of each detection object and geometric parameters of a detection bounding box of the detection object; The three-dimensional target detection model is trained by the method according to any one of claims 1 to 5.

Images