A 3D target detection method based on multiple views

Abstract

This invention discloses a multi-view-based 3D object detection method. The method inputs a target image into a trained detection model to output predicted target object information. The detection model includes an image encoder, an image decoder, and a prediction head. The training process of the detection model is as follows: S1, construct a training set and input it into the image encoder to output multi-view features; S2, input the multi-view features into the image decoder, and perform spatial and temporal aggregation decoding based on a spatiotemporal aggregation module to obtain high-level feature information; S3, input the high-level feature information into the prediction head to obtain the final predicted target object information; S4, calculate the loss based on the predicted information and label information, and optimize the detection model through backpropagation. This object detection method can achieve both high accuracy and high efficiency in multi-view 3D object detection.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a three-dimensional target detection method based on multiple views. Background Technology

[0002] Multi-view 3D object detection is a key technology with wide applications in fields such as autonomous driving, robot navigation, and augmented reality. Currently, technologies in these fields primarily rely on visual sensors to detect and locate objects using images acquired from different perspectives. While some progress has been made, existing technologies still face several challenges and limitations.

[0003] A major problem is localization error, as depth estimation using a single image is difficult, leading to uncertainty in predicting the 3D position of an object. The incompleteness of monocular vision data makes it difficult for existing methods to accurately predict object positions. To address this, some existing methods introduce large image backbone networks or additional depth estimation networks to extract depth information, thereby improving the accuracy of object localization. However, this strategy increases computational complexity, resulting in larger models and reduced inference speed, which is not suitable for practical applications.

[0004] Another challenge is that existing technologies primarily focus on static visual data, neglecting the importance of temporal series information. This makes them ill-suited for scenarios requiring consideration of temporal correlations, such as traffic flow analysis or moving object tracking. Existing technologies have limitations in temporal modeling and spatiotemporal feature aggregation in multi-view 3D object detection, necessitating more flexible approaches to address these challenges.

[0005] Therefore, there is a trade-off between accuracy and computational efficiency in existing technologies for multi-view 3D object detection. Summary of the Invention

[0006] Based on the technical problems existing in the background technology, this invention proposes a three-dimensional target detection method based on multiple views, which can achieve both high accuracy and high efficiency in multi-view three-dimensional object detection.

[0007] The present invention proposes a three-dimensional target detection method based on multiple views, which feeds the target image into a trained detection model to output target object prediction information;

[0008] The detection model includes an image encoder, an image decoder, and a prediction head;

[0009] The training process of the detection model is as follows:

[0010] S1. Construct a training set and input the training set into the image encoder to output multi-view features;

[0011] S2. Input the multi-view features into the image decoder, and perform spatial and temporal aggregation decoding on the multi-view features based on the spatiotemporal aggregation module to obtain high-level feature information;

[0012] S3. Input high-level feature information into the prediction head to obtain the final target object prediction information.

[0013] S4. Calculate the loss based on the prediction information and label information, and optimize the detection model through backpropagation.

[0014] Furthermore, in step S2, when spatially aggregating the multi-view features based on the spatiotemporal aggregation module, the specific steps include:

[0015] Initialize a set of objects to query Q = {q1, q2, ..., q} M}, query q from each object i Decode the reference point c from ∈Q i Reference point c i Considering the i-th predicted bounding box as its center point, the camera projection parameters and camera transformation matrix T are used... n , for each reference point c i The image is projected back onto the camera.

[0016] A linear network is used to predict the k neighbor nodes for each query, and these k+1 nodes are used to form an undirected graph to obtain the query q. i The set of nodes in the connected graph N(i);

[0017] Projecting the set of connected graph nodes N(i) onto the two-dimensional plane of the original target image for feature sampling, we obtain the feature of each graph node v in the set of connected graph nodes N(i). j Hidden state x j ;

[0018] Object Query Based on Multilayer Perceptron i With q i Predicted one-hop neighbor graph node v j The edge weight w between i，j Based on the hidden state x j With edge weight w i，j The product yields a space-based object query scheme.

[0019] Furthermore, after obtaining each query q i After connecting the set of nodes N(i) of the graph, construct the graph G, specifically as follows:

[0020] Based on the fact that each object query only involves reference point features, automatically learned graph connections are obtained, and all graph connections in the set of connected graph nodes N(i) are merged to construct graph G. Each object query adopts a convolutional neural network structure as a unary model for object queries to enhance features.

[0021] Furthermore, the formula for the set of nodes N(i) in the connected graph is as follows:

[0022] N(i) = c i +Δ i =c i +f(q i )

[0023] Among them, c i Indicates the reference point, Δ i This represents the predicted walking path connecting the nodes of the graph, and f(·) represents the multilayer perceptron used to query q for each object. i Generate the transformation offset of the sampling node;

[0024] Each graph node v j Hidden state x j It is expressed as follows:

[0025]

[0026] r jn =T n ·v j v j ∈N(i)

[0027] Where L represents the level of the multi-scale feature map, N represents the number of cameras, and σ jnl The binary value r represents whether the reference point is projected onto the two-dimensional plane of the original target image. jn R represents the nth camera reference point. j The pixel coordinates r projected onto the two-dimensional plane of the original target image. jnl F represents the two-dimensional pixel coordinates of the reference point rj in the nth camera projected onto the feature map of the lth layer. ln Let f(·) represent the feature map of the l-th layer of the n-th image. bilinear T represents the bilinear interpolation function. n This represents the projection matrix of the nth camera;

[0028] The specific spatial object query scheme is as follows:

[0029]

[0030] in, This represents the object query at iteration t+1. Let G(·) represent the object query after iteration t, G(·) represent the graph formed by the reference points, A represent the adjacency matrix of graph G, and W represent the weight matrix of the edge between two nodes in graph G.

[0031] Furthermore, in step S2, when performing temporal aggregation on the multi-view features based on the spatiotemporal aggregation module, the specific steps include:

[0032] In the current time frame t0, the multi-view features are spatially aggregated based on the spatiotemporal aggregation module to construct a spatial graph, which is the graph G constructed at time frame t0;

[0033] At the new time frame t, based on the self-motion velocity of the current frame and the self-vehicle change matrix, the position of the reference point of the current frame in the new time frame t is calculated, and a new spatial map is constructed;

[0034] Two spatial graphs at different times are merged into one graph. The spatiotemporal aggregation module is used to extract features from multiple frames and multi-scale images to obtain the hidden state of each query.

[0035] The hidden state of each query is decoded to obtain high-level feature information.

[0036] Furthermore, multi-frame, multi-scale images are images resulting from the spatial aggregation of continuously moving images by an encoder and a spatiotemporal aggregation module. The formula for calculating the distance traveled between adjacent frames in the three-dimensional coordinate system within a multi-frame, multi-scale image is as follows:

[0037]

[0038]

[0039] Where Δt represents the time interval between two adjacent frames. This indicates the displacement of the vehicle between two consecutive frames. This represents the vehicle's velocity along the x-axis. This indicates the vehicle's velocity along the y-axis. This represents the velocity of the vehicle along the z-axis. This represents the coordinates of the reference point at time t, considering only the displacement of the vehicle. Indicates the coordinates of the reference point at time t0;

[0040] The time- and space-based object query scheme is as follows:

[0041]

[0042] in, This represents the object query at iteration t+1. This represents the object query after iteration t. The reference point c represents time frame t. i coordinates This represents the feature obtained from the feature map at time t through bilinear interpolation. This represents the weight between the i-th query and the j-th node in the graph at time t.

[0043] Furthermore, in step S3, the target object prediction information is represented by a 3D bounding box with classification labels. The parameters of each 3D bounding box include size D = [w, l, h] and pose p = [x, y, z, θ], where θ is the yaw angle of the object, x, y, z represent the center of the box in the lidar coordinate system, and w, l, h represent the width, length and height of the three-dimensional target box.

[0044] Furthermore, the loss function L of the detection model is as follows:

[0045] L = L cls +L reg

[0046]

[0047]

[0048] Among them, L cls L represents the classification loss. reg This represents the regression loss of the target bounding box. These represent the object classification labels and the bounding box labels, respectively. i B i This is to detect the prediction results of the model.

[0049] The advantages of the multi-view-based 3D object detection method provided by this invention are as follows: This method achieves both high accuracy and high efficiency in multi-view 3D object detection. It fully utilizes time-series information to model temporal correlations and effectively aggregate spatiotemporal features, adapting to scenarios that require consideration of object motion and temporal evolution. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the detection model in this invention;

[0051] Figure 2 This is a flowchart of the training process for the detection model. Detailed Implementation

[0052] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] like Figure 1 and 2 As shown, the present invention proposes a three-dimensional target detection method based on multiple views, which inputs the target image into a trained detection model to output target object prediction information.

[0054] Taking a car as an example, a detection model is constructed based on the number, location, and intrinsic parameters of the onboard cameras. This model includes an image encoder, an image decoder, and a prediction head. The image encoder consists of a ResNet backbone and an FPN neck, while the image decoder is a converter-type decoder using a spatiotemporal aggregation module. Specifically, the image encoder comprises a ResNet101 network and an FPN with 4 layers of output features. A student network, specifically a ResNet18, is set up to learn this content. The image decoder has a structure of 256-dimensional input, 512-dimensional output, and an 8-head self-attention interaction module. The prediction head is a two-layer 1x1 convolutional network with 256 channels of input and 17 dimensions (detection box regression: 7 + number of categories: 10) of output.

[0055] The constructed detection model is trained using knowledge distillation techniques. A low-parameter network is used as the student network, mimicking the network portion of the image encoder (containing the ResNet backbone and FPN neck) and the converter-like decoder (including the spatiotemporal aggregation module). This significantly reduces the number of model parameters while maintaining detection model performance, thereby improving subsequent detection efficiency. The training process of the detection model is as follows: Figure 1 and 2 As shown.

[0056] S1. Construct a training set and input the training set into the image encoder to output multi-view features;

[0057] To facilitate training, a corresponding dataset is collected and created based on the specific application scenario. Specifically, data sequences are collected from multiple cameras installed on a car, preprocessed through scaling and cropping, and then input into the image encoder of the detection model. The image encoder in the detection model encodes the input images (images from the training set). Specifically, the images are sequentially input into the ResNet backbone and the FPN neck for image processing. For each image, the FPN neck outputs a feature set F with L layers, and the feature set F within 2 seconds is written to a cache for later retrieval, avoiding redundant computation.

[0058] S2. Input the multi-view features into the image decoder, and perform spatial and temporal aggregation decoding on the multi-view features based on the spatiotemporal aggregation module to obtain high-level feature information;

[0059] The image decoder sequentially performs spatial and temporal aggregation on the input multi-view features to obtain temporal and spatial object query schemes. The query results based on the temporal and spatial object query schemes are then decoded to obtain high-level feature information. The spatial feature aggregation and temporal feature aggregation are detailed below.

[0060] (2-1) Spatial feature aggregation: Spatial aggregation of multi-view features based on the spatiotemporal aggregation module.

[0061] First, initialize a set of object queries Q = {q1, q2, ..., q}. M}, query q from each object i Decode the reference point c from ∈Q i Reference point c i The center point of the i-th predicted bounding box is considered as follows:

[0062] c i =Θ ref (q i )

[0063] Where Θ ref Given an existing linear network, utilize the camera projection parameters and the camera transformation matrix T. n , for each reference point c i The image is projected back into the camera image to obtain location information.

[0064] For a given reference point c i The system treats its spatial neighbors as potential graph nodes and connects them using an adjacency matrix A predicted by the network. It's important to note that the created graph is asymmetric; that is, an edge from node a to b does not imply an edge from b to a. To dynamically sample context-sensitive nodes for each object query, allowing vehicle motion (target image) to move in a uniform space, the system identifies the most informative neighbors for feature aggregation. Assume... It is the predicted traversal path for the connected graph nodes, where k represents the number of neighboring nodes, and each query q i The set of nodes N(i) in the connected graph can be described as:

[0065] N(i) = c i +Δ i =c i +f(q i )

[0066] Here, f is a multilayer perceptron (MLP) used to query q based on each object. i Generate the conversion offset of the sampling node.

[0067] Considering that each object query q i Only reference point features are collected. To improve computational efficiency, edge connections between non-reference points are ignored because each object query only involves reference point features. This results in a graph connection that can be learned automatically. Subsequently, all connections are merged to construct the final graph G, and a convolutional neural network structure is used as a unary model to augment each object query, thus enhancing the features.

[0068] By projecting the neighbors onto a two-dimensional plane of the original target image and performing feature sampling, the feature of each graph node v can be obtained. j Hidden state x j :

[0069]

[0070] r jn =T n ·v j v j ∈N(i)

[0071] Where L represents the level of the multi-scale feature map, N represents the number of surround-view cameras, and σ jnl The binary value r represents whether the reference point is projected onto the two-dimensional plane of the original target image. jn R represents the nth camera reference point. j The pixel coordinates r projected onto the two-dimensional plane of the original target image. jnl Let r represent the reference point in the nth camera. j The two-dimensional pixel coordinates F projected onto the l-th layer feature map ln Let f(·) represent the feature map of the l-th layer of the n-th image. bilinear T represents the bilinear interpolation function. n This represents the projection matrix of the nth camera.

[0072] In practical applications, the target image is a multi-view image, such as images of the same object from different directions. The target image, containing multiple images, is input into an image encoder, which outputs multi-view features containing the same number of images (e.g., if 6 images are input, the image encoder will still output 6 images). These multi-view features are then input into an image decoder for spatial aggregation. In this aggregation, each image in the multi-view features undergoes l layers of feature processing, resulting in a corresponding feature map for each layer. Therefore, when projecting reference points onto the two-dimensional plane of the original target image, each reference point is projected onto the feature map of each layer, yielding two-dimensional pixel coordinates r. jnl By collecting the 2D pixel coordinates corresponding to all layers, the nth camera reference point r is obtained. j The pixel coordinates r projected onto the two-dimensional plane of the original target image jn Therefore, r jn It is layer L r jnl The resulting array has a number that corresponds to the level L of the multi-scale feature map, where the multi-scale feature map corresponds to the multi-view feature map.

[0073] Object Query Based on Multilayer Perceptron i With q i Predicted one-hop neighbor graph node v j The edge weight w between i,j Based on the hidden state x j With edge weight w i,j The product yields a spatial object query scheme. Given that the 3D graph already contains spatial encoding, such graph connections not only include appearance information but also augmented geometric information. In iteration t+1, the final spatial object query scheme is as follows:

[0074]

[0075] in, This represents the object query at iteration t+1. This represents the object query after iteration t. The reference point c represents time frame t. i coordinates This represents the feature obtained from the feature map at time t through bilinear interpolation. The edge weight w represents the weight between the i-th query and the j-th node in the graph at time t. i,j Determined based on a multilayer perceptron (MLP) layer. The proposed spatiotemporal aggregation module spatially aggregates multi-view features (spatial graph model), enabling each object query to fuse instance-specific context and geometric cues for accurate 3D localization.

[0076] (2-2) Temporal graph feature aggregation: Time aggregation of multi-view features based on the spatiotemporal aggregation module.

[0077] In the current time frame t0, based on the self-motion velocity and the vehicle change matrix of the current frame, calculate the position of the reference point of the current frame in the new time frame t, and construct a new spatial map;

[0078] Here, based on the self-motion velocity of the current frame and the self-vehicle change matrix, it is shown that the target image (vehicle) moves over time. The acquired motion video stream, from time t0 to time t, shows the distance the vehicle travels. Calculated using the following formula:

[0079]

[0080] Where Δt represents the time interval between two adjacent frames. This indicates the displacement of the vehicle between two consecutive frames. This represents the vehicle's velocity along the x-axis. This indicates the vehicle's velocity along the y-axis. This represents the velocity of the vehicle along the z-axis. This represents the coordinates of the reference point at time t, considering only the displacement of the vehicle. Let t0 represent the coordinates of the reference point, and c is the coordinate of the reference point.

[0081] Based on the distance traveled, the reference point at time frame t Transform to the three-dimensional coordinate system of time frame t to obtain the reference frame;

[0082]

[0083] in, This represents the coordinates of the reference point at time t. and These are the rotation and translation matrices of the vehicle from the current frame to the target frame t, respectively.

[0084] Two spatial graphs at different times are merged into one graph. The spatiotemporal aggregation module is used to extract features from multiple frames and scales of images to obtain the hidden state of each query.

[0085] The hidden state for each query is as follows:

[0086]

[0087] in, This represents the object query at iteration t+1. This represents the object query after iteration t. The reference point c represents time frame t. icoordinates This represents the feature obtained from the feature map at time t through bilinear interpolation. This represents the weight between the i-th query and the j-th node in the graph. The hidden state of each query is decoded to obtain high-level feature information.

[0088] S3. Input high-level feature information into the prediction head to obtain the final target object prediction information;

[0089] The target object prediction information is represented by a 3D bounding box with classification labels. The parameters of each 3D bounding box include size D = [w, l, h] and attitude p = [x, y, z, θ], where θ is the yaw angle of the object, x, y, z represent the center of the box in the lidar coordinate system, and w, l, h represent the width, length and height of the 3D target box.

[0090] S4. Calculate the loss based on the prediction information and label information, and backpropagate to optimize and train the detection model;

[0091] The loss function L of the detection model in this embodiment is as follows:

[0092] L = L cls +L reg

[0093]

[0094]

[0095] Among them, L cls L represents the classification loss. reg This represents the target bounding box regression loss. These represent the object classification labels and the bounding box labels, respectively. i V i This is to detect the prediction results of the model.

[0096] Based on steps S1 to S3, high accuracy and high efficiency can be achieved in multi-view 3D object detection. By fully utilizing time-series information, modeling of temporal correlations and effective aggregation of spatiotemporal features are realized to adapt to scenarios that require consideration of object motion and temporal evolution.

[0097] To verify the superiority of the multi-view-based 3D object detection method described in this embodiment, tests were conducted on the nuScenes dataset and the Waymo Open Dataset, respectively.

[0098] The nuScenes dataset is one of the most popular large-scale datasets for 3D object detection. It contains 1000 different scenes, each captured by 6 cameras to cover the entire viewpoint. The dataset is officially divided into 700 / 150 / 150 scenes for training / validation / testing. There are a total of 23 classes, of which only 10 are used to compute the final metric. Our model is evaluated using officially predefined metrics, including nuScenes Detection Score (NDS), Mean Accuracy (mAP), Mean Translation Error (mATE), Mean Scale Error (mASE), Mean Orientation Error (mAOE), Mean Velocity Error (mAVE), and Mean Attribute Error (mAAE). The NDS score is a comprehensive evaluation metric that integrates the other metrics and is defined as follows:

[0099]

[0100] The final test and comparison results are shown in Table 1.

[0101] Table 1

[0102]

[0103] Additionally, the Waymo Open Dataset is currently the largest autonomous driving dataset. It contains 798 scenes for training and 202 scenes for validation. It consists of three classes: vehicles, pedestrians, and cyclists. Unlike the nuScenes dataset, it contains only five pinhole cameras with resolutions of 1920×1280 or 1920×886 pixels, covering approximately 250 degrees of field of view. Therefore, 3D labels not captured by the cameras will be removed during evaluation. A mean accuracy based on longitudinal fault-tolerant IoU (LET-IoU) (LET-3D-AP) and a mean accuracy weighted by localization affinity were employed. Specifically, LET-IoU measures the ground reality and the prediction. Based on LET-IoU, LET3D-AP measures the mean accuracy of the prediction through a binary matching with the ground reality. The final results are shown in Table 2.

[0104] Table 2

[0105]

[0106] Based on the comparison of data in Tables 1 and 2, it can be intuitively concluded that the detection method proposed in this embodiment can achieve both high accuracy and high efficiency in multi-view 3D object detection.

[0107] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A three-dimensional target detection method based on multiple views, characterized in that, The target image is fed into a trained detection model to output target object prediction information; The detection model includes an image encoder, an image decoder, and a prediction head; The training process of the detection model is as follows: S1. Construct a training set and input the training set into the image encoder to output multi-view features; S2. Input the multi-view features into the image decoder, and perform spatial and temporal aggregation decoding on the multi-view features based on the spatiotemporal aggregation module to obtain high-level feature information; S3. Input high-level feature information into the prediction head to obtain the final target object prediction information; S4. Calculate the loss based on the prediction information and label information, and optimize the detection model through backpropagation; In step S2, when spatially aggregating the multi-view features based on the spatiotemporal aggregation module, the specific steps include: Initialize a set of object queries Query from each object Decoding the reference point Reference point Considered as the first The center point of each predicted bounding box is determined using camera projection parameters and the camera transformation matrix. , each reference point The image is projected back onto the camera. Predicting each query using a linear network. Each neighboring node, and this The nodes form an undirected graph, and each query is obtained. Connection graph node set ; Connect the graph node set Feature sampling is performed on the two-dimensional plane projected onto the original target image to obtain a set of connected graph nodes. Each graph node Hidden state ; Object Query Determination Based on Multilayer Perceptron with by Predicted one-hop neighbor graph nodes Edge weights between Based on hidden state With edge weight The product yields a space-based object query scheme; In step S2, when performing temporal aggregation on the multi-view features based on the spatiotemporal aggregation module, the specific steps include: In the current time frame In this process, a spatial graph is constructed by spatially aggregating multi-view features based on a spatiotemporal aggregation module. The spatial graph is a time frame. Graphics constructed in time ; New time frame At that time, based on the self-motion velocity and the self-vehicle change matrix of the current frame, the reference point of the current frame is calculated in the new time frame. The location within the space is determined, and a new spatial map is constructed. Two spatial graphs at different times are merged into one graph. The spatiotemporal aggregation module is used to extract features from multiple frames and multi-scale images to obtain the hidden state of each query. The hidden state of each query is decoded to obtain high-level feature information.

2. The three-dimensional target detection method based on multiple views according to claim 1, characterized in that, In obtaining each query Connection graph node set Then, construct the graph. Specifically: Based on the fact that each object query only involves the features of the reference point, automatically learned graph connections are obtained, and the set of connected graph nodes is merged. All graphs in the graph are connected to construct a graph. .

3. The three-dimensional target detection method based on multiple views according to claim 2, characterized in that, Connect graph node set The formula is as follows: in, Indicates a reference point. This represents the predicted walking path connecting the nodes in the graph. This represents a multilayer perceptron, used for querying based on each object. Generate the transformation offset of the sampling node; Each graph node Hidden state It is expressed as follows: in, Represents the hierarchy of multi-scale feature maps. Indicates the number of cameras. This is a binary value indicating whether the reference point is projected onto the two-dimensional plane of the original target image. Indicates the first Camera reference points The pixel coordinates projected onto the two-dimensional plane of the original target image. Indicates the first Reference point in each camera Projected to the Two-dimensional pixel coordinates on the layer feature map Indicates the first The first picture Layer feature map, This represents a bilinear interpolation function. Indicates the first Projection matrix of each camera; The specific spatial object query scheme is as follows: in, Iteration This time object query Iteration This time object query A diagram representing the points referenced. Representing graphics The adjacency matrix, Representing graphics The weight matrix of the edge between two nodes.

4. The three-dimensional target detection method based on multiple views according to claim 1, characterized in that, Multi-frame, multi-scale images are images formed by spatial aggregation of continuously moving images through an encoder and a spatiotemporal aggregation module. The formula for calculating the distance of movement between adjacent frames in a three-dimensional coordinate system in a multi-frame, multi-scale image is as follows: in, This represents the time interval between two adjacent frames. This indicates the displacement of the vehicle between two consecutive frames. Indicates the edge of the car The speed of the shaft's movement Indicates the edge of the car The speed of the shaft movement, Indicates the edge of the car The speed of the shaft movement, express The coordinates of the reference point are considered only when the displacement of the vehicle is taken into account. express The coordinates of the reference point at any given time; The hidden states of each query are as follows: in, Iteration This time object query Iteration This time object query Represents time frame Reference point below coordinates express Features obtained from the feature map at any time through bilinear interpolation express Time of the first The first query and the first The weights between nodes in the graph.

5. The three-dimensional target detection method based on multiple views according to claim 1, characterized in that, In step S3, the target object prediction information is represented by 3D bounding boxes with classification labels. The parameters of each 3D bounding box include size. and posture ,in It is the yaw angle of the object. This represents the center of the box in the lidar coordinate system. This represents the width, length, and height of the 3D target bounding box.

6. The three-dimensional target detection method based on multiple views according to claim 1, characterized in that, In step S4, the loss function of the detection model is... as follows: in, Represents classification loss. This represents the regression loss of the target bounding box. , These represent the object category labels and the bounding box labels, respectively. This is to detect the prediction results of the model.

Images