Three-dimensional target detection method and device, detection model training method and device and electronic equipment

Abstract

The embodiment of the invention provides a three-dimensional target detection method, a detection model training method, a detection model training device and electronic equipment, and the three-dimensional target detection method comprises the steps: carrying out the M times of iteration processing of image features of a to-be-detected target image based on preset object query and initial three-dimensional positions of a plurality of reference points, generating final query features, and performing target detection based on the final query feature and the final three-dimensional position of the reference point to obtain a target detection result. Wherein after each iteration from the first iteration to the Mth iteration, the three-dimensional positions of the plurality of reference points are updated based on the query features obtained by the current processing. Therefore, the reference points are dynamically updated for M times based on the query features generated by each iteration, so that the distribution of the reference points is gradually aligned with the distribution of the real target, the accuracy of the target detection result is effectively improved, and meanwhile, the false detection of the background region is inhibited.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to a three-dimensional target detection method, a training method for the detection model, an apparatus, and an electronic device. Background Technology

[0002] In applications such as autonomous driving, mobile robots, smart transportation, intelligent security, AR (Augmented Reality), and VR (Virtual Reality), using machine vision technology to detect 3D targets based on camera images is a crucial step in realizing these functions.

[0003] Traditional camera-based 3D object detection methods can be broadly categorized into two types:

[0004] 1) Bird's-eye view-based method: This method explicitly projects camera image features into the bird's-eye view space and performs dense predictions on the bird's-eye view plane. The advantage of this method is its clear geometric consistency, but it suffers from high computational and memory overhead due to view transformation error accumulation and dense grid inference.

[0005] 2) Sparse query-based approach: This approach directly aggregates sparse object queries with camera image features, avoiding the construction of dense bird's-eye view features. Therefore, it is more efficient and suitable for real-time applications. The core of this paradigm is that each query consists of a content embedding (or object query) and a location embedding generated from reference points. These reference points act as geometric anchors, guiding camera feature sampling and driving attention.

[0006] Currently, most sparse query-based methods employ a fixed reference point design: during the initial training phase, a set of reference points is initialized randomly / through clustering, allowing the object detection model to learn the distribution of these reference points autonomously; during inference, the distribution of these reference points remains unchanged across all scenes and decoding layers. This approach results in approximately uniformly distributed reference points, making it difficult to distinguish between foreground and background, leading to an increase in high-confidence false positives in background regions. Summary of the Invention

[0007] The purpose of this application is to provide a three-dimensional target detection method, a detection model training method, an apparatus, and an electronic device to reduce false detections and improve detection accuracy. The specific technical solution is as follows:

[0008] This application provides a three-dimensional target detection method, including:

[0009] Obtain the image of the target to be detected;

[0010] Extract image features from the target image to be detected;

[0011] Based on a preset object query and the initial 3D positions of multiple preset reference points, the image features are processed M times to generate the final query features; wherein, the second to the Mth iterations are based on the intermediate query features generated in the previous iteration and the updated 3D positions of the multiple reference points, and the updated 3D positions of the multiple reference points are updated based on the intermediate query features obtained in this processing after each of the first to the (M-1)th iterations; M>2;

[0012] The three-dimensional positions of multiple reference points in the (M-1)th update are updated based on the final query features to obtain the three-dimensional positions of multiple reference points after this update, which are used as the final three-dimensional positions of the reference points.

[0013] Target detection is performed based on the final query features and the three-dimensional position of the final reference point to obtain the target detection results.

[0014] In some embodiments, the extraction of image features from the target image to be detected and subsequent steps are implemented by a pre-trained 3D target detection model; the 3D target detection model includes: a target prediction sub-model and a reference point optimization sub-model; the target prediction sub-model includes: an M-layer first neural network based on an attention mechanism; the reference point optimization sub-model includes a second neural network;

[0015] The image feature extraction and subsequent steps for the target image to be detected specifically include:

[0016] The target image to be detected, the preset object query, and the initial 3D positions of multiple preset reference points are input into the target prediction sub-model; and the initial 3D positions of the multiple reference points are input into the reference point optimization sub-model.

[0017] The target prediction sub-model extracts image features from the target image to be detected and outputs the image features to each layer of the first neural network;

[0018] The first layer of the first neural network performs self-attention and cross-attention interaction based on the image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on a preset object query and the initial three-dimensional position of multiple preset reference points;

[0019] In the second to Mth layers of the first neural network, each layer performs self-attention and cross-attention interaction based on the image features and the queries of the second to Mth layers, so that the second to M-1th layers output intermediate query features and the Mth layer outputs the final query features; wherein, the queries of the second to Mth layers are generated based on the intermediate query features output by the previous layer and the updated three-dimensional positions of multiple reference points output by the reference point optimization sub-model;

[0020] The target prediction sub-model performs target detection based on the final query features output by the Mth layer and the three-dimensional position of the final reference point output by the reference point optimization sub-model, and obtains the target detection result.

[0021] The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network; wherein, the 3D positions of the multiple reference points updated in the (M-1)th update are updated based on the final query features output by the Mth layer, and the updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points.

[0022] In some embodiments, the reference point optimization sub-model includes: M scene adaptive refinement modules; wherein, the M scene adaptive refinement modules are respectively connected to the first to the Mth layers of the first neural network;

[0023] The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network, including:

[0024] Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference point and the query features output by a layer of the corresponding first neural network. This process yields the 3D offsets of multiple reference points output by the second neural network. The 3D offsets are then fused with the 3D positions of the multiple reference points before the update to obtain the updated 3D positions of each reference point. The 3D positions of the reference points input to the first scene adaptive refinement module are the initial 3D positions of each reference point. The 3D positions of the reference points input to the second to Mth scene adaptive refinement modules are the updated 3D positions of each reference point output by the previous scene adaptive refinement module.

[0025] In some embodiments, each scene adaptive refinement module includes: a position encoding module, a second neural network, and an optimization module; the second neural network includes: an offset regression branch and a position-aware branch;

[0026] Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and intermediate query features to obtain the 3D offsets of multiple reference points output by the second neural network, including:

[0027] The position encoding module is used to encode the three-dimensional position of the input reference point into a position vector;

[0028] The position vector is processed by the position-aware branch to obtain the position-aware vector;

[0029] The query features are processed using the offset regression branch to output the 3D offsets of multiple reference points; during the processing, the intermediate feature vector of the offset regression branch is fused with the position-aware vector.

[0030] The step of fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update includes: the optimization module fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update.

[0031] In some embodiments, the 3D target detection model is obtained in advance through joint training of the target prediction sub-model and the reference point optimization sub-model. In some embodiments, the M-layer first neural network is an M-layer Transformer decoding network; the second neural network is a lightweight neural network.

[0032] This application embodiment also provides a training method for a three-dimensional object detection model, the three-dimensional object detection model including: an object prediction sub-model and a reference point optimization sub-model; the object prediction sub-model including: an M-layer first neural network based on an attention mechanism; the reference point optimization sub-model including a second neural network; the training method including:

[0033] Obtain the 3D position of the target ground truth bounding boxes of all target objects in the labeled sample image;

[0034] Based on the 3D position information of the target ground truth bounding boxes of each target object in the sample image, scene-level reference point pseudo-ground truth values ​​of the 3D position of the reference point are generated.

[0035] The sample image, the preset object query, and the initial 3D positions of multiple preset reference points are input into the target prediction sub-model; and the initial 3D positions of the multiple reference points are input into the reference point optimization sub-model.

[0036] The target prediction sub-model extracts sample image features from the sample image and outputs the sample image features to each layer of the first neural network.

[0037] The first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on a preset object query and the initial three-dimensional position of multiple preset reference points;

[0038] In the second to Mth layers of the first neural network, each layer performs self-attention and cross-attention interaction based on the sample image features and the queries of the second to Mth layers, so that the second to M-1 layers output intermediate query features and the Mth layer outputs the final query features; wherein, the queries of the second to Mth layers are generated based on the intermediate query features output by the previous layer and the updated three-dimensional positions of multiple reference points output by the reference point optimization sub-model;

[0039] The target prediction sub-model performs target detection based on the final query features output by the Mth layer and the three-dimensional position of the final reference point output by the reference point optimization sub-model, and obtains the sample target prediction box.

[0040] The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network; wherein, the 3D positions of the multiple reference points updated in the (M-1)th update are updated based on the final query features output by the Mth layer, and the updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points.

[0041] The calculation of the total model loss includes: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the final reference point 3D position and the scene-level reference point pseudo-ground truth; and calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss.

[0042] Based on the total loss of the model, the network parameters of the first neural network and the second neural network are updated to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0043] In some embodiments, generating scene-level reference point pseudo-ground values ​​for the 3D position of reference points based on the 3D position information of the target ground truth bounding boxes of each target object in the sample image includes:

[0044] Obtain the geometric center coordinates of the target ground truth bounding boxes of all target objects from the labeled sample images;

[0045] Obtain the geometric dimensions of the target truth boxes for all target objects from the labeled sample images;

[0046] For each target object, an anisotropic independent Gaussian distribution is formed based on the geometric center coordinates and geometric dimensions of its target truth box;

[0047] Sampling is performed in the above Gaussian distribution to generate a set of reference points;

[0048] Obtain the 3D position information corresponding to each reference point and save it as the scene-level reference point pseudo-true value.

[0049] In some embodiments, the reference point optimization sub-model includes: M scene adaptive refinement modules; wherein, the M scene adaptive refinement modules are respectively connected to the first to the Mth layers of the first neural network;

[0050] The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network, including:

[0051] Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and the query features output by a layer of the corresponding first neural network. This process yields the 3D offsets of multiple reference points output by the second neural network. The 3D offsets are then fused with the 3D positions of the reference points before the update to obtain the updated 3D positions of each reference point. The 3D positions of the reference points input to the first scene adaptive refinement module are the initial 3D positions of each reference point. The 3D positions of the reference points input to the second to Mth scene adaptive refinement modules are the updated 3D positions of each reference point output by the previous scene adaptive refinement module.

[0052] The step of updating the network parameters of the first neural network and the second neural network based on the total loss of the model includes:

[0053] Based on the total loss of the model, the network parameters of each layer of the first neural network are updated, and the network parameters of the second neural network in each scene adaptive refinement module are updated.

[0054] In some embodiments, each scene adaptive refinement module includes: a position encoding module and a second neural network; the second neural network includes: a position-aware branch and an offset regression branch;

[0055] Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and intermediate query features to obtain the 3D offsets of multiple reference points output by the second neural network, including:

[0056] The position encoding module is used to encode the three-dimensional position of the input reference point into a position vector;

[0057] The position vector is processed by the position-aware branch to obtain the position-aware vector;

[0058] The query features are processed using the offset regression branch to output the 3D offsets of multiple reference points; during the processing, the intermediate feature vector of the offset regression branch is fused with the position-aware vector.

[0059] The step of updating the network parameters of the second neural network in each scene adaptive refinement module includes updating the network parameters of the position-aware branch and the offset regression branch.

[0060] In some embodiments, calculating the target prediction loss based on the sample target prediction bounding box and the target ground truth bounding box includes: calculating the target prediction loss using the following loss function. :

[0061]

[0062] in y represents the truth value, b represents the target truth box, and c represents the corresponding category; , Indicates the test results. Represents the target prediction box and Corresponding category For the focus of loss, For norm one loss, The optimal matching function between the predicted bounding box and the ground truth bounding box of the sample target; These are preset hyperparameters;

[0063] The calculation of the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-true values ​​includes: calculating the reference point distribution loss using the following loss function. :

[0064]

[0065] Where G represents the pseudo-real value of a scene-level reference point, g represents the pseudo-real value of a reference point; P represents the optimized reference point P obtained by the scene adaptive refinement module, and p represents the optimized value of a reference point.

[0066] This means: for each predicted reference point, find the nearest false true value point, calculate the squared Euclidean distance, and take the average. This means that for each pseudo-true value point, the nearest predicted reference point is found, and the average of the squared Euclidean distance is calculated; where α and β are preset hyperparameters.

[0067] in and The distance weighting factor is calculated using the following formula:

[0068] The calculation of the total model loss for this training based on the target prediction loss and the reference point distribution loss includes: calculating the total model loss using the following loss function. :

[0069] .

[0070] This application embodiment also provides a three-dimensional target detection device, including: an image acquisition module for acquiring an image of a target to be detected; a feature extraction module for extracting image features from the image of the target to be detected; an iterative processing module for performing M iterations on the image features based on a preset object query and the initial three-dimensional positions of a preset plurality of reference points to generate final query features; wherein the second to the Mth iterations are processing the image features based on intermediate query features generated in the previous iteration and the updated three-dimensional positions of the plurality of reference points; wherein the updated three-dimensional positions of the plurality of reference points are updated based on the intermediate query features obtained in this processing after each of the first to the (M-1)th iterations; M>2; and a detection module for performing target detection based on the final query features to obtain a target detection result.

[0071] This application embodiment also provides a training device for a 3D object detection model, including: a ground truth acquisition module, used to obtain the 3D positions of the ground truth boxes of all target objects in an annotated sample image; a pseudo ground truth acquisition module, used to generate scene-level reference point pseudo ground truth values ​​of the 3D positions of reference points based on the 3D position information of the ground truth boxes of each target object in the sample image; a joint training module, used to input the sample image, a preset object query, and the preset initial 3D positions of multiple reference points into the target prediction sub-model; and input the initial 3D positions of the multiple reference points into the reference point optimization sub-model; the target prediction sub-model, for... The sample image extracts sample image features, and outputs these features to each layer of the first neural network. The first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first-layer query, outputting the first-layer intermediate query features. The first-layer query is generated based on a preset object query and the initial 3D positions of multiple preset reference points. The second to Mth layers of the first neural network each perform self-attention and cross-attention interaction based on the sample image features and the second to Mth layer queries, such that the second to M-1th layers output intermediate query features, and the Mth layer outputs the final query features. The Mth layer query is generated based on the intermediate query features output from the previous layer and the updated 3D positions of multiple reference points output by the reference point optimization sub-model. The target prediction sub-model performs target detection based on the final query features output from the Mth layer and the final 3D positions of the reference points output by the reference point optimization sub-model to obtain sample target prediction boxes. The reference point optimization sub-model uses the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output from each layer of the first neural network. Specifically, the 3D positions of the multiple reference points updated in the (M-1)th iteration are updated based on the final query features output from the Mth layer. The process involves updating the 3D positions of multiple reference points to obtain the final 3D positions of the reference points; and calculating the total model loss, including: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-ground truth; calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss; and updating the network parameters of the first neural network and the second neural network based on the total model loss to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0072] This application also provides an electronic device, including: a memory for storing computer programs; and a processor for executing the program stored in the memory to implement the above-described target detection method or the above-described training method for a three-dimensional target detection model.

[0073] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described target detection method or the above-described training method for a three-dimensional target detection model.

[0074] Beneficial effects of the embodiments in this application:

[0075] This application provides a three-dimensional target detection method, a training method for a detection model, an apparatus, and an electronic device. The three-dimensional target detection method involves performing M iterations on the image features of the target image to be detected, based on a preset object query and the initial three-dimensional positions of multiple reference points, to generate final query features. Target detection is then performed based on the final query features and the final three-dimensional positions of the reference points to obtain the target detection result.

[0076] In each of the first to Mth iterations, the 3D positions of multiple reference points are updated based on the query features obtained in this processing. In this way, the reference points are dynamically updated M times based on the query features generated in each iteration, so that the distribution of the reference points is successively aligned with the distribution of the real target, thereby effectively improving the accuracy of the target detection results and suppressing false detections in the background area.

[0077] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0079] Figure 1 This is a flowchart of a three-dimensional target detection method provided in an embodiment of this application.

[0080] Figure 2 This is a schematic diagram of the structure of a three-dimensional detection model used to implement the three-dimensional target detection method provided in the embodiments of this application.

[0081] Figure 3 The flowchart shows the overall scheme for implementing the three-dimensional target detection method provided in the embodiments of this application based on a three-dimensional target detection model.

[0082] Figure 4 for Figure 2 The flowchart shown illustrates the process of generating pseudo-true values ​​for scene-level reference points in the overall scheme.

[0083] Figure 5 for Figure 2 The flowchart shown is a diagram illustrating the training process of the 3D target detection model in the overall scheme.

[0084] Figure 6 This is a schematic diagram of the structure of the three-dimensional target detection device provided in the embodiments of this application.

[0085] Figure 7 This is a schematic diagram of the structure of the training device for the three-dimensional target detection model provided in the embodiments of this application.

[0086] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0087] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0088] To reduce false detections and improve detection accuracy, embodiments of this application provide a three-dimensional target detection method, a detection model training method, an apparatus, and an electronic device, which will be described in detail below.

[0089] The three-dimensional target detection method provided in this application embodiment is referred to [reference needed]. Figure 1 , Figure 1 This is a flowchart illustrating a three-dimensional target detection method provided in an embodiment of this application. Figure 1 As shown, the process includes:

[0090] Step S101: Obtain the image of the target to be detected;

[0091] Step S102: Extract image features from the image of the target to be detected;

[0092] Step S103: Based on a preset object query and the initial 3D positions of multiple preset reference points, the image features are processed M times to generate the final query features; wherein, the second to the Mth iterations are based on the intermediate query features generated in the previous iteration and the updated 3D positions of multiple reference points, and the updated 3D positions of multiple reference points are updated based on the intermediate query features obtained in this processing after each of the first to M-1 iterations; M>2;

[0093] Step S104: Update the three-dimensional positions of multiple reference points in the (M-1)th update based on the final query features to obtain the three-dimensional positions of multiple reference points after this update, which are used as the final three-dimensional positions of the reference points.

[0094] Step S105: Target detection is performed based on the final query features and the three-dimensional position of the final reference point to obtain the target detection result.

[0095] In this embodiment, the reference points are dynamically updated M times based on the intermediate query features generated in each iteration, aligning the distribution of the reference points with the distribution of the real targets. This effectively improves the accuracy of target detection results while suppressing false detections in background areas. Therefore, the 3D target detection method provided in this embodiment can be widely applied to scenarios with high real-time and high-precision requirements for 3D target perception, such as autonomous driving, mobile robots, intelligent transportation, intelligent security, and AR / VR.

[0096] In some embodiments, the preset object query can be a vector initialized to all zeros, whose dimensions are consistent with the positional encoding of the initial 3D positions of multiple reference points. The preset initial 3D positions of multiple reference points are the initial position coordinates of N reference points in the image, where N is typically 600-900. M can be set according to actual needs, for example, it can be 6, 8, or more times, and this application embodiment does not impose any limitations.

[0097] In some embodiments, steps S102 to S105 described above can be implemented by a pre-trained 3D object detection model. See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a 3D detection model used to implement the 3D target detection method provided in the embodiments of this application. Solid lines represent variables, and dashed lines represent processing modules.

[0098] like Figure 2 As shown, the 3D target detection model includes: a target prediction sub-model 100 and a reference point optimization sub-model 200; the target prediction sub-model 100 includes: an M-layer first neural network based on an attention mechanism, for example: Figure 2 The diagram shows an M-layer Transformer decoding network; a reference point optimization sub-model 200, including a second neural network, which may include... Figure 2 The offset regression branch 211 and the position-aware branch 212 are shown.

[0099] like Figure 2 As shown, the process of realizing 3D target detection using this 3D target detection model includes the following steps:

[0100] Step 1: Input the target image to be detected, the preset object query, and the initial 3D positions of multiple preset reference points into the target prediction sub-model 100; and input the initial 3D positions of the multiple reference points into the reference point optimization sub-model; wherein, the target image to be detected ( Figure 2(The camera images in the image) In autonomous driving scenarios, these can be V-shaped loop view images or single-lane front view images of the vehicle.

[0101] Step 2, the target prediction sub-model extracts image features from the image of the target to be detected and outputs the image features to each layer of the first neural network;

[0102] Step 3: The first layer of the first neural network performs self-attention and cross-attention interaction based on image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on the preset object query and the initial 3D position of multiple preset reference points;

[0103] Step 4: In the second to Mth layers of the first neural network, each layer performs self-attention and cross-attention interaction based on image features and queries from the second to Mth layers, so that the second to M-1th layers output intermediate query features and the Mth layer outputs the final query features; wherein, the queries from the second to Mth layers are generated based on the intermediate query features output by the previous layer and the updated 3D positions of multiple reference points output by the reference point optimization sub-model.

[0104] Step 5: The target prediction sub-model performs target detection based on the final query features output from layer M and the 3D position of the final reference point output from the reference point optimization sub-model 200, obtaining the target detection result; such as... Figure 2 The detection module performs target detection based on the final query features output from the Mth layer and the 3D position of the final reference point output from the reference point optimization sub-model 200, and obtains the target prediction box.

[0105] Step 6: Optimize the reference point sub-model. Using the second neural network, based on the query features output by each layer of the first neural network, update the 3D positions of multiple reference points layer by layer. Specifically, based on the final query features output by the Mth layer, update the 3D positions of multiple reference points updated in the (M-1)th update. The updated 3D positions of multiple reference points are then used as the final 3D positions of the reference points.

[0106] In this embodiment, 3D target detection is achieved through a pre-trained 3D target detection model, resulting in faster detection speed. Furthermore, by using a reference point optimization sub-model, the query features output from the previous layer of the first neural network are used to progressively update and optimize the reference points.

[0107] In some existing technologies, iterative updates of reference points can be achieved between decoding layers of a neural network. However, the supervision only comes from the final detection head of the neural network, rather than optimizing the reference points independently. The update method involves using the detection head to detect the features output by each decoding layer of the neural network, obtaining the target prediction box for each layer, and using the target prediction box of each layer as the reference point for the next layer's decoding iteration. In this existing approach, because only a small number of reference points falling near the real target boxes receive effective supervision, although the reference points are optimized during inference, their distribution still does not match the distribution of the real target. This leads to a highly uneven target distribution (e.g., in intelligent transportation scenarios such as intersections, occlusions, and differences in distance scales), where the background region is still prone to high-confidence false detections.

[0108] In this embodiment, a reference point optimization sub-model is used to independently update and optimize the reference points layer by layer based on the query features output by the previous layer of the first neural network, targeting the initial 3D positions of multiple reference points. This reference point optimization sub-model can be jointly trained with the target prediction sub-model. During model training, an additional reference point distribution loss can be introduced to explicitly supervise the movement of all reference points, thereby effectively aligning the distribution of reference points with that of the target.

[0109] like Figure 2 As shown, the first neural network in this embodiment can be an M-layer Transformer decoding network; using the transformer structure as the decoder, geometric information is introduced using reference points, and the query features are refined through the interaction of object query and image features. After M iterations, the final query features are obtained.

[0110] In some embodiments, the target prediction sub-model 100 first encodes the initial 3D positions of multiple reference points using a reference point position encoding module to obtain reference point position codes. Then, the reference point position codes are fused with the object query to obtain a first-level query. The dimension of the object query is the same as the dimension of the reference point position codes. There are two fusion methods: one is to add the reference point position codes to the object query to obtain the first-level query, in which case the dimensions of the first-level query, the object query, and the reference point position codes are all the same; the other is to concatenate the reference point position codes with the object query to obtain the first-level query, in which case the dimension of the first-level query is twice the dimension of the object query and the reference point position codes.

[0111] In this embodiment, the second neural network may include a bias regression branch 211 and a position-aware branch 212. Both branches can be lightweight neural networks, such as SqueezeNet, ShuffleNet, MobileNet series, FBNet, etc. These models reduce the amount of computation and parameters by optimizing the structural design.

[0112] In some existing technologies, to compensate for the common defect of scene-level misalignment between reference points and targets, a common approach is to increase the number of reference points, making them densely distributed throughout the detection space. However, this results in a significant increase in computational power consumption. In this embodiment, a lightweight neural network is used to continuously and explicitly guide the distribution of reference points to align with the distribution of scene objects in each decoding layer, thereby improving detection performance and robustness without significantly increasing parameters and computational power. Compared with the mainstream sparse query method in existing technologies, the embodiment of this application has the dual advantages of lightweight design and high accuracy.

[0113] like Figure 2 As shown, the reference point optimization sub-model 200 in this embodiment includes: M scene adaptive refinement modules 210 ( Figure 2 (Only two are shown in the image). Among them, M scene adaptive refinement modules 210 are connected one-to-one with the first to the Mth layers of the M-layer Transformer decoding network.

[0114] Thus, each scene adaptive refinement module 210 uses the offset regression branch 211 and the position awareness branch 212 of the second neural network to process the input 3D position of the reference point and the intermediate query features output by the corresponding Transformer decoding network layer, to obtain the 3D offset of multiple reference points output by the offset regression branch 211, and then fuses the 3D offset with the 3D position of multiple reference points before the update to obtain the updated 3D position of each reference point; wherein, the 3D position of the reference point input to the first scene adaptive refinement module 210 is the initial 3D position of each reference point; the 3D positions of the reference points input to the second to Mth scene adaptive refinement modules 210 are the updated 3D positions of each reference point output by the previous scene adaptive refinement module 210.

[0115] In this embodiment, the reference point optimization sub-model 200 includes M scene adaptive refinement modules 210, each of which is connected to the first to the Mth layers in a one-to-one correspondence.

[0116] In this embodiment, the network parameters trained by the offset regression branch 211 of each scene adaptive refinement module 210 are different, thus achieving a gradual decrease in offset as the reference point is updated and optimized layer by layer. After M-1 optimizations by the scene adaptive refinement module, the reference point gradually clusters towards the dense target region, serving as one of the inputs to the next layer transformer for self-attention and cross-attention interactions. After the reference point is updated by the Mth scene adaptive refinement module 210, the final 3D position of the reference point is obtained, making the final 3D position of the reference point for target detection more effectively aligned with the distribution of the target object, further improving the detection accuracy.

[0117] like Figure 2 As shown, in this embodiment, each scene adaptive refinement module 210 includes: a position encoding module, for example: Figure 2 The sine and cosine coding module 213 and the second neural network ( Figure 2 The system comprises a migration regression branch 211, a position-aware branch 212, and an optimization module 214. In this embodiment, each scene adaptive refinement module 210 uses a sine / cosine encoding module 213 to encode the 3D position of the input reference points into a position vector. The position vector is processed by the position-aware branch 212 to obtain a position-aware vector. The migration regression branch 211 processes the query features and outputs the 3D migrations of multiple reference points. During processing, the intermediate feature vector of the migration regression branch 211 is fused with the position-aware vector obtained by the position-aware branch 212, injecting global distribution information of the reference points. Finally, the optimization module 214 fuses the 3D migrations with the 3D positions of the multiple reference points before the update to obtain the updated 3D positions of each reference point. For example, the migration regression branch 211 can be implemented based on an MLP (Multilayer Perceptron). After the query features are input into the MLP, each hidden layer outputs an intermediate feature vector. Each intermediate feature vector is fused with the position-aware vector and used as the input to the next layer of the MLP, ultimately outputting the 3D migrations of multiple reference points.

[0118] like Figure 2 As shown, in this embodiment, the sine-cosine encoding module 213 encodes the three-dimensional coordinates of the reference point into a sine-cosine position vector, and maps it to a high-dimensional feature space through a position-aware branch 212 (e.g., another lightweight network) to obtain a position-aware vector. For example, the position-aware branch 212 can also be implemented based on an MLP.

[0119] In this embodiment, the query features are input into the offset regression branch 211. The intermediate feature vector generated during the processing of the offset regression branch 211 is fused with the position-aware vector output by the position-aware branch 212, and finally the three-dimensional offset of multiple reference points is output.

[0120] In this embodiment, the position-aware branch 212 enables the global distribution information of reference points to be injected into the process of predicting the three-dimensional offset of multiple reference points using the offset regression branch 211. This allows the offset regression branch 211 to predict the offset of multiple reference points while having a global view of the reference point distribution, ensuring that the updated distribution of reference points is aligned with the distribution of the real target.

[0121] Depend on Figure 2 As can be seen, each scene adaptive refinement module 210 has two inputs: a. the query features output by the current transformer decoding layer, typically an N×D dimensional variable, where N is the number of targets to be predicted and D is the feature dimension; and b. the set of geometric coordinates of the current reference point (the first scene adaptive refinement module 210 inputs the initial reference point; the second to Mth scene adaptive refinement modules 210 input the reference points after the first to M-1th updates, respectively) in three-dimensional space, typically an N×3 dimensional variable, where N is the number of targets to be predicted. In this embodiment, N three-dimensional reference points are preset, and both the query (input) and query feature (output) dimensions are N, ultimately predicting N target boxes.

[0122] The scene adaptive refinement module 210 in this embodiment includes two branches: a. a position-aware branch: encoding the three-dimensional coordinates of reference points into sine-cosine position vectors and mapping them to a high-dimensional feature space through a lightweight network to obtain position-aware vectors; and b. a offset regression branch: obtaining the offsets of the three-dimensional coordinates of multiple reference points using query features. The intermediate feature vector of the offset regression branch is fused with the position-aware vector output by the position-aware branch to finally output the three-dimensional offsets of multiple reference points. In this embodiment, there can be multiple ways to fuse the query features with the position-aware vector (such as sequentially adding the position-aware vector output by the position-aware branch to the intermediate feature vector of the offset regression branch), and this embodiment does not impose any limitations.

[0123] Furthermore, in this embodiment, the position-aware branch 212 in the scene adaptive refinement module 210 uses a lightweight network that can be designed to share parameters across all transformer decoding layers. This means that the network parameters of the position-aware branch 212 in each scene adaptive refinement module 210 are identical. This ensures a small number of system parameters, avoiding model bloat due to increased layer count; it also ensures stable training, maintaining consistency in the update patterns of reference points at different levels; and it has strong transferability, allowing for easy embedding into different baseline models.

[0124] like Figure 2As shown, the optimization module 214 in this embodiment can perform information fusion using an adder to obtain the optimized reference point. Thus, in this embodiment, the scene adaptive refinement module 210 achieves intra-layer position awareness and dynamic updating of reference points through cross-layer parameter sharing in the position awareness branch, enabling the reference point to gradually approximate the real target distribution in each transformer decoding stage.

[0125] like Figure 2 As shown, in this embodiment, the backbone network in the target prediction sub-model 100 can extract image features from the target image to be detected and output the image features to the first neural network ( Figure 2 The backbone network consists of M layers of a Transformer decoding network. Different models can be selected based on task requirements.

[0126] Simultaneously, the detection module in the target prediction sub-model 100 can perform target detection based on the final query features output from the Mth layer and the final reference point 3D position output from the Mth scene adaptive refinement module 210, thus obtaining the target detection result. Figure 2 The target prediction bounding box (identifying the predicted target center position, size, angle, velocity, etc.) and the corresponding target prediction category.

[0127] In this embodiment, the initial 3D positions of multiple preset reference points are first encoded as reference point position codes and mapped to high-dimensional position vectors. These are then combined with a preset object query to generate a first-layer query. This query enters the first-layer Transformer decoding network and interacts with image features through self-attention and cross-attention. Each optimized 3D position of a reference point is first encoded as a reference point position code and mapped to a high-dimensional position vector. This vector is then combined with the query features output from the previous layer's Transformer decoding network to generate a query for this layer. This query then enters the current layer's Transformer decoding network and interacts with image features through self-attention and cross-attention. The high-dimensional position vector has the same dimension as the query feature data output by the Transformer decoding network. In this embodiment, the M-layer Transformer decoding network can have 6, 8, or more layers; this application embodiment does not impose any limitations.

[0128] like Figure 2As shown, in this embodiment, the optimization module 214 in the first scene adaptive refinement module 210 of the reference point optimization sub-model 200 fuses the 3D offsets of multiple reference points output by the first offset regression branch 211 with the initial 3D positions of the multiple reference points to generate optimized 3D positions of the multiple reference points. The optimization module 214 in the second scene adaptive refinement module 210 fuses the 3D offsets of multiple reference points output by the second offset regression branch 211 with the optimized 3D positions of the multiple reference points output by the optimization module 214 of the first scene adaptive refinement module 210 to generate optimized 3D positions of the multiple reference points. This achieves layer-by-layer optimization of the reference points. Figure 2 As shown, the optimization module 214 in this embodiment can perform fusion using an adder. In other embodiments, the optimization module 214 can also employ a gating network or an attention network (e.g., another Transformer network) to achieve fusion. The specific fusion method is not limited in this embodiment.

[0129] The 3D object detection model in this embodiment is obtained in advance through joint training of the object prediction sub-model and the reference point optimization sub-model. The overall scheme for implementing 3D object detection using this model is described in detail below. See [link to documentation]. Figure 3 , Figure 3 This is a flowchart illustrating the overall scheme for implementing the 3D target detection method provided in the embodiments of this application based on a 3D target detection model. Figure 3 As shown, the scheme mainly includes three steps: construction of scene-level reference point pseudo-real values, training of the 3D object detection model, and inference of the 3D object detection model.

[0130] In this embodiment, the construction of scene-level reference point pseudo-ground values ​​can be achieved through the following two steps: Step 1, obtain the 3D positions of the target ground truth boxes of all target objects in the labeled sample images; Step 2, based on the 3D position information of the target ground truth boxes of each target object in the sample images, generate scene-level reference point pseudo-ground values ​​for the 3D positions of the reference points. Figure 3 As shown, we can first obtain the target center coordinates and target size (i.e., the three-dimensional position of the target truth box) of all target objects in the sample image from the target truth value, and then use the target center coordinates and target size information to generate reference point pseudo-truth values.

[0131] The ground truth of the target includes a 3D bounding box and the target category. The 3D bounding box contains attributes such as velocity, and the center coordinates and dimensions of the bounding box identify the target's position and size, respectively. Using this bounding box, a scene reference point pseudo-ground truth is constructed to explicitly supervise the movement of the reference points during training. Each unit of the constructed scene-level reference point pseudo-ground truth consists of the coordinates of n 3D points, where n is a pre-defined number of object queries, i.e., whether n represents the number of targets to be predicted.

[0132] For details, see Figure 4 , Figure 4 for Figure 2 The flowchart shown illustrates the process of generating scene-level reference point pseudo-truth values ​​in the overall scheme. This process includes:

[0133] Step S401: Obtain the geometric center coordinates of the target ground truth boxes of all target objects from the labeled sample images;

[0134] Step S402: Obtain the geometric dimensions of the target ground truth boxes of all target objects from the labeled sample images;

[0135] Step S403: For each target object, an anisotropic independent Gaussian distribution is generated based on the geometric center coordinates and geometric dimensions of its target truth box.

[0136] Step S404: Sample from the above Gaussian distribution to generate a set of reference points;

[0137] Step S405: Obtain the three-dimensional position information corresponding to each reference point and save it as the scene-level reference point pseudo-true value.

[0138] In this embodiment, the target ground truth includes a 3D bounding box and a target category. The 3D bounding box contains attributes such as velocity, and the center coordinates and dimensions of the bounding box respectively identify the target's position and size. Therefore, after completing the training of the 3D target detection model, inputting the image to be detected (such as a V-channel loop view image or a single-channel front view image) will directly yield information such as the position, size, orientation, category, and velocity of the 3D target. In this embodiment, the geometric dimensions of the target ground truth box refer to the length, width, and height of the 3D target ground truth box.

[0139] Specifically, step S403 above can be: for each target object, the mean is the geometric center coordinates of its target truth box, and the variances of the length, width, and height of its target truth box are used as the variances of each coordinate axis, forming an anisotropic independent Gaussian distribution.

[0140] Step S404 above can specifically be: performing uniform random sampling within the Gaussian distribution to generate a set of pseudo-target points. The number of sampling points is usually roughly consistent with the number of object queries to ensure the balance of alignment supervision.

[0141] In other embodiments, the sampling operation for constructing scene-level reference point pseudo-truth values ​​can be weighted according to the volume, category, etc. of the target object, and this application does not limit it.

[0142] Step S405 above can involve saving these scene-level reference point pseudo-values ​​offline after sampling. These scene-level reference point pseudo-values ​​can be used to calculate the reference point distribution loss during the training of the 3D object detection model, thereby guiding the global alignment of the optimized reference points with the real object distribution.

[0143] like Figure 3 As shown, in this embodiment, the main process of training the 3D object detection model is as follows: Sample images (such as camera images) and the 3D positions of various references pre-defined object queries and initializations are input into the 3D object detection model. Based on the target prediction bounding boxes output by the target prediction sub-model in the 3D object detection model and the pre-defined ground truth bounding boxes, the classification and regression loss values ​​are calculated using pre-defined classification and regression loss functions. Simultaneously, based on the optimized reference points and scene-level reference point pseudo-ground truth values ​​output by the reference point optimization sub-model in the 3D object detection model, the reference point distribution loss value is calculated using a pre-defined reference point distribution loss function. The classification and regression loss values ​​and the reference point distribution loss value are added together to obtain the total model loss. Then, based on the total model loss, the 3D object detection model is trained under supervision. Specifically, the first neural network of the M layers in the target prediction sub-model (such as...) can be trained under supervision. Figure 2 The M-layer Transformer decoding network in the model and the second neural network in the reference point optimization sub-model (such as...) Figure 2 The parameters of the offset regression branch 211 and the position-aware branch 212 in the model are adjusted until the total loss function of the model converges.

[0144] For details, see Figure 5 , Figure 5 for Figure 2 The diagram shows the training flowchart of the 3D object detection model in the overall scheme. Figure 5 As shown, the training process includes:

[0145] Step S501: Input the sample image, the preset object query, and the initial 3D positions of multiple preset reference points into the target prediction sub-model; and input the initial 3D positions of the multiple reference points into the reference point optimization sub-model;

[0146] Step S502: The target prediction sub-model extracts sample image features from the sample image and outputs the sample image features to each layer of the first neural network.

[0147] Step S503: The first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on a preset object query and the initial three-dimensional position of multiple preset reference points;

[0148] Step S504: In the second to M layers of the first neural network, each layer performs self-attention and cross-attention interaction based on the sample image features and the queries of the second to M layers, so that the second to M-1 layers output intermediate query features and the M layer outputs the final query features; wherein, the queries of the second to M layers are generated based on the intermediate query features output by the previous layer and the updated three-dimensional positions of multiple reference points output by the reference point optimization sub-model.

[0149] Step S505: The target prediction sub-model performs target detection based on the final query features output by the Mth layer and the three-dimensional position of the final reference point output by the reference point optimization sub-model, and obtains the sample target prediction box.

[0150] Step S506: The reference point optimization sub-model uses the second neural network to update the three-dimensional positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network; wherein, the three-dimensional positions of multiple reference points updated for the (M-1)th time are updated based on the final query features output by the Mth layer, and the updated three-dimensional positions of multiple reference points are used as the final three-dimensional positions of the reference points.

[0151] Step S507, calculate the total model loss, including: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the final reference point 3D position and the scene-level reference point pseudo-ground truth; and calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss.

[0152] Step S508: Based on the total loss of the model, update the network parameters of the first neural network and the second neural network to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0153] like Figure 5 As shown in the flowchart, the training process of this 3D target detection model achieves global explicit supervision, that is, the reference point distribution loss explicitly supervises the movement of all reference points, which improves the inherent defect of traditional methods that rely solely on the final detection loss, resulting in sparse supervision.

[0154] The 3D object detection model trained in this embodiment can be... Figure 2 The 3D object detection model shown is trained in this way, enabling foreground clustering and background suppression. This means that reference points can cluster towards the high-density foreground region layer by layer, while avoiding overactivation in the background region.

[0155] like Figure 2 As shown, the reference point optimization sub-model 200 of the 3D object detection model trained in this embodiment may include: M scene adaptive refinement modules 210; wherein, the M scene adaptive refinement modules 210 are respectively connected to the first to the Mth layers of the first neural network. The structure and function of each scene adaptive refinement module 210 in the reference point optimization sub-model 200 are as described above, and will not be repeated here.

[0156] In this embodiment, the reference point optimization sub-model 200 includes multiple scene adaptive refinement modules 210, and each scene adaptive refinement module 210 contains a second neural network, such as... Figure 2 The first neural network includes a second neural network with a offset regression branch 211 and a position-aware branch 212. Therefore, during training, based on the total loss of the model, the network parameters of each layer of the first neural network are updated, and the network parameters of the offset regression branch 211 and the position-aware branch 212 in the second neural network of each scene adaptive refinement module, as well as the network parameters of the backbone network, are also updated.

[0157] In this embodiment, after training, the network parameters of the offset regression branch 211 of each scene adaptive refinement module 210 are different, thereby realizing that the offset decreases layer by layer as the reference points are updated and optimized. In this way, after the reference points are optimized by the scene adaptive refinement module M times, they gradually gather towards the dense target area, so that the distribution of reference points used by the Mth layer Transformer decoding network is more effectively aligned with the distribution of targets, further improving the detection accuracy.

[0158] against Figure 2 The training process of the 3D object detection model shown may specifically include:

[0159] Input sample images (such as V-channel loop view images or single-channel front view images). Figure 2 The camera images are input into the backbone network, which extracts multi-scale sample image features and outputs them to each layer of the M-layer transformer decoding network.

[0160] Initialize N 3D reference points and encode their positions. Combine these with content embedding (i.e., a pre-defined object query) to form a query, which is then input into the first layer of an M-layer transformer decoding network. Simultaneously, input the initial 3D positions of the reference points into the first scene adaptive refinement module.

[0161] The first layer of the transformer decoding network performs object query and self-attention and cross-attention interaction of image features, and outputs the query features of the first layer to the first scene adaptive refinement module.

[0162] like Figure 2 As shown, the first scene adaptive refinement module processes the initial 3D position of the input initial reference point and the query features output from the first layer of the M-layer transformer decoding network, and outputs the 3D offset of multiple reference points. The 3D offset is added to the initial 3D position by an adder to obtain the optimized 3D position of the reference point.

[0163] The optimized 3D position of the reference point, after being encoded and mapped in a high dimension, is combined with the query features output from the first layer of the transformer decoding network to generate a second-layer query, which is then input into the second layer of the transformer decoding network.

[0164] Layers 2 through M of the transformer decoding network perform self-attention and cross-attention interactions based on sample image features and queries from layers 2 through M. This results in layers 2 through M-1 outputting intermediate query features, and layer M outputting the final query feature. The intermediate query features from layers 2 through M-1 are then fed to the next layer and the next scene adaptive refinement module of the transformer decoding network for further transformer decoding and reference point optimization, achieving layer-by-layer refinement of query features and layer-by-layer updating of reference points. The final query feature from layer M is fed to the Mth scene adaptive refinement module to update the 3D positions of multiple reference points updated in the M-1th iteration, obtaining the final 3D positions of the reference points after the Mth update.

[0165] In this way, the final query features output by the Mth layer of the transformer decoding network and the final reference point 3D position output by the Mth scene adaptive refinement module are input into the detection module to obtain the target prediction bounding box (identifying the position and size of the predicted target) and the target prediction classification output by the detection module.

[0166] Then, the total model loss can be calculated, including: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-ground truth values; and calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss. Based on the total model loss, the network parameters of the first neural network and the second neural network are updated to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0167] The target prediction loss can be calculated using the following loss function. :

[0168] (Formula 1)

[0169] in y represents the truth value, b represents the target truth box, and c represents the corresponding category; y' represents the detection result, b' represents the target prediction box, and c' represents the category. For the focus of loss, For norm one loss, The optimal matching function between the predicted bounding box and the ground truth bounding box of the sample target; These are preset hyperparameters;

[0170] Calculate the reference point distribution loss using the following loss function. :

[0171] (Formula 2)

[0172] Where G represents the pseudo-real value of a scene-level reference point, g represents the pseudo-real value of a reference point; P represents the optimized reference point P obtained by the scene adaptive refinement module, and p represents the optimized value of a reference point.

[0173] This means: for each predicted reference point, find the nearest false true value point, calculate the squared Euclidean distance, and take the average. This means that for each pseudo-true value point, the nearest predicted reference point is found, and the average of the squared Euclidean distance is calculated; where α and β are preset hyperparameters.

[0174] in and The distance weighting factor is calculated using the following formula: (Formula 3)

[0175] in, and It is a point In the normalized coordinates in the horizontal direction, i is the index of the reference point;

[0176] The total loss of the model is calculated using the following loss function. :

[0177] (Formula 4).

[0178] This embodiment is applied in a vehicle autonomous driving scenario, where the distance weighting factor can decrease exponentially with the distance to the vehicle, emphasizing safety-related targets around the vehicle. In other embodiments, the distance weighting factor may employ other strategies such as segmentation, which are not limited in this application.

[0179] As can be seen from the loss function above, the reference point distribution loss not only ensures that the optimized reference points are aligned with the global distribution of the target, but also strengthens the focus on key targets in the near field, significantly improving detection robustness and practical value.

[0180] like Figure 3 As shown, in this embodiment, the main inference process of the 3D object detection model is as follows: After the 3D object detection model training is completed, the target image to be detected (such as a camera image) and the 3D positions of various references preset by object query and initialization are input into the 3D object detection model, and the 3D object detection model outputs the target prediction box. For example, after the 3D object detection model training is completed, a V-channel surround view image or a single-channel front view image is input, and the 3D object detection model can directly obtain information such as the position, size, orientation, category, and velocity of the 3D target based on the 3D positions of various references preset by object query and initialization.

[0181] In summary, the 3D target prediction model applied in this application demonstrates stable gains across various backbone networks and paradigms. Furthermore, it exhibits stronger generalization capabilities for target imbalance scenarios such as intersections and occlusions in applications like autonomous driving and intelligent transportation, effectively reducing systematic false detections. Therefore, it can be widely applied to sparse query-based target detection methods, providing stable gains and lower migration costs, with particularly excellent performance improvement in target imbalance scenarios.

[0182] This application provides a method for training a three-dimensional target detection model, including... Figure 3 The process of generating scene-level reference point pseudo-ground values ​​and training the 3D object detection model are shown below. For details, please refer to [link / reference]. Figure 4 and Figure 5 The detailed explanation will not be repeated here.

[0183] Corresponding to the aforementioned three-dimensional target detection method, this application also provides a three-dimensional target detection device, see [link to relevant documentation]. Figure 6 , Figure 6This is a schematic diagram of the structure of the three-dimensional target detection device provided in an embodiment of this application. Figure 6 As shown, the device includes:

[0184] Image acquisition module 601 is used to acquire an image of the target to be detected;

[0185] Feature extraction module 602 is used to extract image features from the target image to be detected;

[0186] The iterative processing module 603 is used to perform M iterations on the image features based on a preset object query and the initial 3D positions of multiple preset reference points to generate the final query features; wherein, the second to the Mth iterations are performed on the image features based on the intermediate query features generated in the previous iteration and the updated 3D positions of the multiple reference points; wherein, the updated 3D positions of the multiple reference points are updated based on the intermediate query features obtained in this processing after each of the first to the (M-1)th iterations; M>2;

[0187] The final reference point 3D position acquisition module 604 is used to update the 3D positions of multiple reference points updated in the (M-1)th update based on the final query features, and obtain the 3D positions of multiple reference points after this update as the final reference point 3D positions;

[0188] The detection module 605 is used to perform target detection based on the final query features and obtain the target detection results.

[0189] Corresponding to the aforementioned training method for a 3D object detection model, this application also provides a training device for a 3D object detection model, see [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram of the structure of a training device for a three-dimensional target detection model provided in an embodiment of this application. Figure 7 As shown, the training device includes:

[0190] The truth value acquisition module 701 is used to obtain the three-dimensional position of the target truth bounding boxes of all target objects in the labeled sample image;

[0191] The pseudo-truth value acquisition module 702 is used to generate scene-level reference point pseudo-truth values ​​of the three-dimensional position of the reference point based on the three-dimensional position information of the target truth boxes of each target object in the sample image.

[0192] The joint training module 703 is used to input the sample image, a preset object query, and the initial 3D positions of multiple preset reference points into the target prediction sub-model; and to input the initial 3D positions of the multiple reference points into the reference point optimization sub-model; the target prediction sub-model extracts sample image features from the sample image and outputs the sample image features to each layer of the first neural network; the first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on the preset object query and the initial 3D positions of multiple preset reference points; the second to M layers of the first neural network each perform self-attention and cross-attention interaction based on the sample image features and the second to M layer queries, so that the second to M- The system outputs intermediate query features at layer 1 and final query features at layer M. The queries at layers 2 through M are generated based on the intermediate query features output from the previous layer and the updated 3D positions of multiple reference points output by the reference point optimization sub-model. The target prediction sub-model performs target detection based on the final query features output from layer M and the final 3D positions of the reference points output by the reference point optimization sub-model, obtaining sample target prediction boxes. The reference point optimization sub-model uses the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output from each layer of the first neural network. Specifically, the 3D positions of the multiple reference points updated at the (M-1)th update are updated based on the final query features output from layer M, and the resulting updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points.

[0193] The calculation of the total model loss includes: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-ground truth; calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss; and updating the network parameters of the first neural network and the second neural network based on the total model loss to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0194] This application also provides an electronic device, such as... Figure 8 As shown, it includes:

[0195] Memory 801 is used to store computer programs;

[0196] When processor 802 executes a program stored in memory 801, it performs the following steps:

[0197] Obtain the target image to be detected; extract image features from the target image; based on a preset object query and the initial 3D positions of multiple preset reference points, perform M iterations on the image features to generate the final query features; wherein, the second to Mth iterations are performed on the image features based on the intermediate query features generated in the previous iteration and the updated 3D positions of the multiple reference points; wherein, the updated 3D positions of the multiple reference points are updated based on the intermediate query features obtained in this processing after each of the first to M-1th iterations; M>2; perform target detection based on the final query features to obtain the target detection result. Alternatively, implement the following steps:

[0198] Obtain the 3D position of the target ground truth bounding boxes of all target objects in the labeled sample image; based on the 3D position information of the target ground truth bounding boxes of each target object in the sample image, generate scene-level reference point pseudo-ground truth values ​​of the 3D position of the reference point;

[0199] The sample image, a preset object query, and the initial 3D positions of multiple preset reference points are input into the target prediction sub-model; the initial 3D positions of the multiple reference points are input into the reference point optimization sub-model; the target prediction sub-model extracts sample image features from the sample image and outputs the sample image features to each layer of the first neural network; the first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on the preset object query and the initial 3D positions of multiple preset reference points; the second to M layers of the first neural network each perform self-attention and cross-attention interaction based on the sample image features and the second to M layer queries, so that the second to M-1 layers output intermediate query features, and the M layer outputs the final query features; wherein, the second to M layer queries are generated based on the intermediate query features output by the previous layer and the updated 3D positions of multiple reference points output by the reference point optimization sub-model; the target prediction sub-model extracts sample image features from the sample image and outputs the initial ... The first neural network performs target detection based on the final query features output from the Mth layer and the final reference point 3D position output from the reference point optimization sub-model, obtaining sample target prediction boxes. The reference point optimization sub-model uses the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output from each layer of the first neural network. Specifically, the 3D positions of multiple reference points updated in the (M-1)th update are based on the final query features output from the Mth layer, and the updated 3D positions of the multiple reference points are used as the final reference point 3D positions. The model's total loss is calculated, including: calculating the target prediction loss based on the sample target prediction boxes and the target ground truth boxes; calculating the reference point distribution loss based on the final reference point 3D positions and the scene-level reference point pseudo-ground truths; calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss; and updating the network parameters of the first and second neural networks based on the total model loss to jointly train the target prediction sub-model and the reference point optimization sub-model.

[0200] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 802, communication interface, and memory 801 communicating with each other via the communication bus.

[0201] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0202] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0203] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0204] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0205] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described three-dimensional target detection methods or training methods for three-dimensional target detection models.

[0206] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the three-dimensional object detection methods or training methods of the three-dimensional object detection model in the above embodiments.

[0207] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0208] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0209] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0210] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A three-dimensional target detection method, characterized in that, include: Obtain the image of the target to be detected; Extract image features from the target image to be detected; Based on a preset object query and the initial 3D positions of multiple preset reference points, the image features are processed M times to generate the final query features; wherein, the second to the Mth iterations are based on the intermediate query features generated in the previous iteration and the updated 3D positions of the multiple reference points, and the updated 3D positions of the multiple reference points are updated based on the intermediate query features obtained in this processing after each of the first to the (M-1)th iterations; M>2; The three-dimensional positions of multiple reference points in the (M-1)th update are updated based on the final query features to obtain the three-dimensional positions of multiple reference points after this update, which are used as the final three-dimensional positions of the reference points. Target detection is performed based on the final query features and the three-dimensional position of the final reference point to obtain the target detection results.

2. The three-dimensional target detection method according to claim 1, characterized in that, The extraction of image features from the target image to be detected and subsequent steps are implemented by a pre-trained 3D target detection model; The three-dimensional target detection model includes: a target prediction sub-model and a reference point optimization sub-model; the target prediction sub-model includes: an M-layer first neural network based on an attention mechanism; the reference point optimization sub-model includes a second neural network; The image feature extraction and subsequent steps for the target image to be detected specifically include: The target image to be detected, the preset object query, and the initial 3D positions of multiple preset reference points are input into the target prediction sub-model; and the initial 3D positions of the multiple reference points are input into the reference point optimization sub-model. The target prediction sub-model extracts image features from the target image to be detected and outputs the image features to each layer of the first neural network; The first layer of the first neural network performs self-attention and cross-attention interaction based on the image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on a preset object query and the initial three-dimensional position of multiple preset reference points; In the second to Mth layers of the first neural network, each layer performs self-attention and cross-attention interaction based on the image features and the queries of the second to Mth layers, so that the second to M-1th layers output intermediate query features and the Mth layer outputs the final query features; wherein, the queries of the second to Mth layers are generated based on the intermediate query features output by the previous layer and the updated three-dimensional positions of multiple reference points output by the reference point optimization sub-model; The target prediction sub-model performs target detection based on the final query features output by the Mth layer and the three-dimensional position of the final reference point output by the reference point optimization sub-model, and obtains the target detection result. The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network; wherein, the 3D positions of the multiple reference points updated in the (M-1)th update are updated based on the final query features output by the Mth layer, and the updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points.

3. The three-dimensional target detection method according to claim 2, characterized in that, The reference point optimization sub-model includes: M scene adaptive refinement modules; wherein, the M scene adaptive refinement modules are respectively connected to the first to the Mth layers of the first neural network; The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network, including: Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference point and the query features output by a layer of the corresponding first neural network. This process yields the 3D offsets of multiple reference points output by the second neural network. The 3D offsets are then fused with the 3D positions of the multiple reference points before the update to obtain the updated 3D positions of each reference point. The 3D positions of the reference points input to the first scene adaptive refinement module are the initial 3D positions of each reference point. The 3D positions of the reference points input to the second to Mth scene adaptive refinement modules are the updated 3D positions of each reference point output by the previous scene adaptive refinement module.

4. The three-dimensional target detection method according to claim 3, characterized in that, Each scene adaptive refinement module includes: a position encoding module, a second neural network, and an optimization module; the second neural network includes: an offset regression branch and a position-aware branch; Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and intermediate query features to obtain the 3D offsets of multiple reference points output by the second neural network, including: The position encoding module is used to encode the three-dimensional position of the input reference point into a position vector; The position vector is processed by the position-aware branch to obtain the position-aware vector; The query features are processed using the offset regression branch to output the 3D offsets of multiple reference points; during the processing, the intermediate feature vector of the offset regression branch is fused with the position-aware vector. The step of fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update includes: the optimization module fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update.

5. The three-dimensional target detection method according to claim 2, characterized in that, The three-dimensional target detection model is obtained in advance through joint training of the target prediction sub-model and the reference point optimization sub-model.

6. The three-dimensional target detection method according to claim 2, characterized in that, The first neural network of the M-layer is an M-layer Transformer decoding network; The second neural network is a lightweight neural network.

7. A training method for a three-dimensional target detection model, characterized in that, The 3D target detection model includes: a target prediction sub-model and a reference point optimization sub-model; the target prediction sub-model includes: an M-layer first neural network based on an attention mechanism; the reference point optimization sub-model includes a second neural network; the training method includes: Obtain the 3D position of the target ground truth bounding boxes of all target objects in the labeled sample image; Based on the 3D position information of the target ground truth bounding boxes of each target object in the sample image, scene-level reference point pseudo-ground truth values ​​of the 3D position of the reference point are generated. The sample image, the preset object query, and the initial 3D positions of multiple preset reference points are input into the target prediction sub-model; and the initial 3D positions of the multiple reference points are input into the reference point optimization sub-model. The target prediction sub-model extracts sample image features from the sample image and outputs the sample image features to each layer of the first neural network. The first layer of the first neural network performs self-attention and cross-attention interaction based on the sample image features and the first layer query, and outputs the first layer intermediate query features; wherein, the first layer query is generated based on a preset object query and the initial three-dimensional position of multiple preset reference points; In the second to Mth layers of the first neural network, each layer performs self-attention and cross-attention interaction based on the sample image features and the queries of the second to Mth layers, so that the second to M-1 layers output intermediate query features and the Mth layer outputs the final query features; wherein, the queries of the second to Mth layers are generated based on the intermediate query features output by the previous layer and the updated three-dimensional positions of multiple reference points output by the reference point optimization sub-model; The target prediction sub-model performs target detection based on the final query features output by the Mth layer and the three-dimensional position of the final reference point output by the reference point optimization sub-model, and obtains the sample target prediction box. The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network; wherein, the 3D positions of the multiple reference points updated in the (M-1)th update are updated based on the final query features output by the Mth layer, and the updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points. The calculation of the total model loss includes: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the final reference point 3D position and the scene-level reference point pseudo-ground truth; and calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss. Based on the total loss of the model, the network parameters of the first neural network and the second neural network are updated to jointly train the target prediction sub-model and the reference point optimization sub-model.

8. The training method for the three-dimensional target detection model according to claim 7, characterized in that, The process of generating scene-level reference point pseudo-ground values ​​for the 3D position of reference points based on the 3D position information of the target ground truth bounding boxes of each target object in the sample image includes: Obtain the geometric center coordinates of the target ground truth bounding boxes of all target objects from the labeled sample images; Obtain the geometric dimensions of the target truth boxes for all target objects from the labeled sample images; For each target object, an anisotropic independent Gaussian distribution is formed based on the geometric center coordinates and geometric dimensions of its target truth box; Sampling is performed in the above Gaussian distribution to generate a set of reference points; Obtain the 3D position information corresponding to each reference point and save it as the scene-level reference point pseudo-true value.

9. The training method for the three-dimensional target detection model according to claim 7, characterized in that, The reference point optimization sub-model includes: M scene adaptive refinement modules; wherein, the M scene adaptive refinement modules are respectively connected to the first to the Mth layers of the first neural network; The reference point optimization sub-model utilizes the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network, including: Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and the query features output by a layer of the corresponding first neural network. This process yields the 3D offsets of multiple reference points output by the second neural network. The 3D offsets are then fused with the 3D positions of the reference points before the update to obtain the updated 3D positions of each reference point. The 3D positions of the reference points input to the first scene adaptive refinement module are the initial 3D positions of each reference point. The 3D positions of the reference points input to the second to Mth scene adaptive refinement modules are the updated 3D positions of each reference point output by the previous scene adaptive refinement module. The step of updating the network parameters of the first neural network and the second neural network based on the total loss of the model includes: Based on the total loss of the model, the network parameters of each layer of the first neural network are updated, and the network parameters of the second neural network in each scene adaptive refinement module are updated.

10. The training method for the three-dimensional target detection model according to claim 9, characterized in that, Each scene adaptive refinement module includes: a position encoding module, a second neural network, and an optimization module; the second neural network includes: a position-aware branch and an offset regression branch; Each scene adaptive refinement module utilizes a second neural network to process the input 3D position of the reference points and intermediate query features to obtain the 3D offsets of multiple reference points output by the second neural network, including: The position encoding module is used to encode the three-dimensional position of the input reference point into a position vector; The position vector is processed by the position-aware branch to obtain the position-aware vector; The query features are processed using the offset regression branch to output the 3D offsets of multiple reference points; during the processing, the intermediate feature vector of the offset regression branch is fused with the position-aware vector. The step of fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update includes: the optimization module fusing the three-dimensional offset with the three-dimensional positions of multiple reference points before the update to obtain the three-dimensional positions of each reference point after the update. The step of updating the network parameters of the second neural network in each scene adaptive refinement module includes updating the network parameters of the position-aware branch and the offset regression branch.

11. The training method for the three-dimensional target detection model according to claim 7, characterized in that, The step of calculating the target prediction loss based on the sample target prediction bounding box and the target ground truth bounding box includes: calculating the target prediction loss using the following loss function. : ; in y represents the truth value, b represents the target truth box, and c represents the corresponding category; , Indicates the test results. Represents the target prediction box Indicates the corresponding category For the focus of loss, For norm one loss, The optimal matching function between the predicted bounding box and the ground truth bounding box of the sample target; These are preset hyperparameters; The calculation of the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-true values ​​includes: calculating the reference point distribution loss using the following loss function. : ; Where G represents the pseudo-real value of a scene-level reference point, g represents the pseudo-real value of a reference point; P represents the optimized reference point P obtained by the scene adaptive refinement module, and p represents the optimized value of a reference point. This means: for each predicted reference point, find the nearest false true value point, calculate the squared Euclidean distance, and take the average. This means that for each pseudo-true value point, the nearest predicted reference point is found, and the average of the squared Euclidean distance is calculated; where α and β are preset hyperparameters. in and The distance weighting factor is calculated using the following formula: ; in, and It is a point In the normalized coordinates in the horizontal direction, i is the index of the reference point; The calculation of the total model loss for this training based on the target prediction loss and the reference point distribution loss includes: calculating the total model loss using the following loss function. : 。 12. A three-dimensional target detection device, characterized in that, include: The image acquisition module is used to acquire the image of the target to be detected; The feature extraction module is used to extract image features from the target image to be detected; An iterative processing module is used to perform M iterations on the image features based on a preset object query and the initial 3D positions of multiple preset reference points to generate the final query features. The second to Mth iterations are performed on the image features based on the intermediate query features generated in the previous iteration and the updated 3D positions of the multiple reference points. The updated 3D positions of the multiple reference points are updated based on the intermediate query features obtained in each of the first to M-1 iterations; M>2. The detection module is used to perform target detection based on the final query features and obtain the target detection results.

13. A training device for a three-dimensional target detection model, characterized in that, include: The truth value acquisition module is used to obtain the 3D position of the target truth bounding boxes of all target objects in the labeled sample image; The pseudo-truth value acquisition module is used to generate scene-level reference point pseudo-truth values ​​of the three-dimensional position of the reference point based on the three-dimensional position information of the target truth boxes of each target object in the sample image. The joint training module is used to input the sample image, the preset object query, and the initial 3D positions of multiple preset reference points into the target prediction sub-model; and to input the initial 3D positions of the multiple reference points into the reference point optimization sub-model. The target prediction sub-model extracts sample image features from the sample image and outputs these features to each layer of the first neural network. The first layer of the first neural network performs self-attention and cross-attention interactions based on the sample image features and the first-layer query, outputting intermediate query features. The first-layer query is generated based on a preset object query and the initial 3D positions of multiple preset reference points. The second to Mth layers of the first neural network each perform self-attention and cross-attention interactions based on the sample image features and the queries from the second to Mth layers, resulting in the second to M-1th layers outputting intermediate query features and the Mth layer outputting the final query features. The queries from the second to Mth layers are based on the previous layer... The target prediction submodel generates a target prediction box based on the final query features output by the Mth layer and the updated 3D positions of multiple reference points output by the reference point optimization submodel. The target prediction submodel performs target detection based on the final query features output by the Mth layer and the final 3D positions of the reference points output by the reference point optimization submodel. The reference point optimization submodel uses the second neural network to update the 3D positions of multiple reference points layer by layer based on the query features output by each layer of the first neural network. Specifically, the 3D positions of the multiple reference points updated in the (M-1)th update are based on the final query features output by the Mth layer, and the updated 3D positions of the multiple reference points are used as the final 3D positions of the reference points. The calculation of the total model loss includes: calculating the target prediction loss based on the sample target prediction box and the target ground truth box; calculating the reference point distribution loss based on the 3D positions of multiple reference points used in the Mth layer iteration of the first neural network and the scene-level reference point pseudo-ground truth; calculating the total model loss for this training based on the target prediction loss and the reference point distribution loss; and updating the network parameters of the first neural network and the second neural network based on the total model loss to jointly train the target prediction sub-model and the reference point optimization sub-model.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the three-dimensional target detection method according to any one of claims 1-6, or the training method of the three-dimensional target detection model according to any one of claims 7-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the three-dimensional target detection method according to any one of claims 1-6 or the training method for the three-dimensional target detection model according to any one of claims 7-11.

Images