Unsupervised domain adaptive monocular three-dimensional target detection method

Abstract

The invention discloses an unsupervised domain adaptive monocular three-dimensional target detection method and system, and the method comprises the steps: constructing a monocular three-dimensional target detection model, and carrying out the pre-training and self-training; monocular three-dimensional target detection is realized by using the trained monocular three-dimensional target detection model; the monocular three-dimensional target detection model comprises a depth map generation module for receiving an original monocular image and outputting a normalized scale depth map containing relative scale depth information; a pseudo laser point cloud generation module which receives the normalized scale depth map and generates a pseudo laser point cloud; the pseudo label generation module receives the pseudo laser point cloud and outputs the pseudo laser point cloud with a pseudo label; the multi-modal fusion module is used for receiving the pseudo laser point cloud with the pseudo label and the original monocular image and performing fusion to obtain fusion features; and the point cloud target detector realizes accurate positioning and classification of objects in a three-dimensional space according to the fusion features. According to the unsupervised domain adaptive strategy, the performance and robustness of the system are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and deep learning, and in particular to an unsupervised domain adaptive monocular three-dimensional target detection method. Background Art

[0002] Three-dimensional object detection is widely used in environmental perception, scene understanding, three-dimensional reconstruction and other fields, and plays a key role in them. Vision-based three-dimensional object detection sensors are divided into monocular and binocular sensors. Compared with binocular three-dimensional object detection, monocular three-dimensional object detection has the advantages of low cost, easy installation, and easy data acquisition. It is a more challenging and promising research direction. Most of the current monocular three-dimensional object detection methods are based on supervised learning methods, which requires a large number of accurate real labels, which is a great challenge. Therefore, the model of unsupervised domain adaptive monocular three-dimensional object detection is closer to practical applications. However, the unsupervised domain adaptive method faces the problem that the performance will be significantly reduced in cross-domain applications.

[0003] At the same time, the pseudo laser point cloud method is an important method in monocular 3D target detection. It makes full use of the depth information obtained by monocular depth estimation to construct a pseudo laser point cloud and completes target detection through a laser point cloud detector. However, this usually requires two stages to complete. The problem is that it cannot fully utilize the correlation between depth estimation and target detection, which will lead to large errors. Summary of the invention

[0004] The purpose of the present invention is to solve the problems in the prior art.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide an unsupervised domain adaptive monocular 3D target detection method, comprising the following steps:

[0006] Build a monocular 3D target detection model and perform training;

[0007] Use the trained monocular 3D target detection model to realize monocular 3D target detection;

[0008] The monocular 3D target detection model includes: a depth map generation module, which adopts a self-supervised monocular depth estimation network to receive the original monocular image, uses the absolute scale recovery technology, combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo laser point cloud generation module, which receives the normalized scale depth map, generates a pseudo laser point cloud through coordinate transformation and post-processing; a pseudo label generation module, which receives the pseudo laser point cloud, generates a pseudo label, and outputs a pseudo laser point cloud with a pseudo label; a multimodal fusion module, which receives the pseudo laser point cloud with a pseudo label and the original monocular image, and fuses them to obtain fusion features; a point cloud target detector, which realizes accurate positioning and classification of objects in three-dimensional space according to the fusion features.

[0009] Preferably, the training includes:

[0010] Pre-training: In the early stages of building a monocular 3D object detection model, a labeled dataset is used as source domain data to train the model so that it can learn the basic features that can recognize and process 3D objects, helping the model capture key visual and depth information.

[0011] Self-training: The pre-trained monocular 3D object detection model is applied to an unlabeled dataset, i.e., target domain data. The model generates a pseudo laser point cloud and generates pseudo labels based on it, and then uses these pseudo labels to further train and optimize the model.

[0012] Preferably, the pre-training includes pre-training of a pseudo-label generation model and pre-training of a point cloud target detector, and the pre-training process adopts an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model and improve the domain adaptation performance of the point cloud target detector.

[0013] Preferably, the object random scaling strategy comprises the following steps:

[0014] For a labeled 3D object bounding box, with size (l, w, h), center coordinates (u, v, d) and orientation angle ω, scaling transformation is performed along the length, width and height of the detection box to convert the point from the global coordinate system to the local coordinate system of the detection box, expressed as:

[0015]

[0016] in, represents the coordinates of the i-th point in the detection box of size (l,w,h), represents matrix multiplication,;

[0017] By random scaling factor (f l ,f w ,f h ) scales the point coordinates within the box to

[0018] Convert the scaled point coordinates back to the global point cloud coordinate system:

[0019]

[0020] Preferably, the self-training includes self-training of a pseudo-label generation module, by introducing a quality-aware three-interval memory library to guide the generation process of the target domain pseudo-label, including the following steps:

[0021] Get the prediction box generated by the point cloud detector;

[0022] The quality of the predicted box is evaluated based on the IoU scoring criterion, which is pre-trained in the source domain based on the object random scaling strategy;

[0023] A three-interval prediction box partitioning scheme is used to guide the memory library update and pseudo-label generation. The memory library is updated based on the fusion of the current pseudo-label and the historical pseudo-label, and the pseudo-label required for subsequent training is continuously updated.

[0024] Preferably, the process of generating a pseudo laser point cloud by the pseudo laser point cloud generating module comprises the following steps:

[0025] The data acquisition process of the laser radar is simulated through coordinate transformation; the projection of the point cloud on the vertical plane is mapped and distributed according to the vertical angle to construct an angle-distance grid; based on the angle-distance grid, a series of scan lines are generated to simulate the scanning of the laser radar and generate a preliminary laser point cloud; each scan line is composed of points with an approximate vertical angle and arranged in order of distance value, and the point closest to the radar is regarded as the first obstacle encountered by the corresponding laser beam;

[0026] The preliminary pseudo laser point cloud is post-processed; pseudo laser radar data points located outside a specified height range in the preliminary pseudo laser point cloud are removed, and the specified height range is preset according to the characteristics of the real laser radar data.

[0027] Preferably, the multimodal fusion module uses a feature weighted aggregation mechanism to fuse the point cloud data with the original monocular image, including the following steps:

[0028] Receiving fused features from outside the candidate region and fused features from inside the candidate region, including point cloud features and image features;

[0029] Use the self-attention mechanism to establish a weight relationship between features at different positions, output weights to weight the original features, and generate weighted features;

[0030] The weighted features are fused to form a feature representation containing interactive information, which is input into the point cloud object detector as the fused features.

[0031] Preferably, before the multimodal fusion module fuses the pseudo-labeled pseudo-laser point cloud with the original monocular image, the pseudo-labeled pseudo-laser point cloud is quantized by a soft quantization method: a radial basis function weighted method is used for calculation; for a given set of point cloud data P = {p1, p2, p3, ..., pN}, a 3D occupancy tensor T is generated by a soft quantization method. softquantization , where each unit u is associated with a fixed center position p c If point p falls outside unit u, T soft quantization (u) = 0; if point p falls within unit u, it is expressed as:

[0032]

[0033] Among them, W(u) is the radial basis function weight, δ represents a very small positive value, represents the neighboring unit in the neighborhood u of unit u; when δ is much larger than 0 and the neighborhood r of unit u u When is an empty set, the above soft quantization returns to the original binary definition.

[0034] The present invention also provides an unsupervised domain adaptive monocular three-dimensional object detection system, comprising:

[0035] Model training module, builds a monocular 3D target detection model and performs training;

[0036] The object detection module uses the trained monocular 3D object detection model to realize monocular 3D object detection;

[0037] The monocular 3D target detection model includes: a depth map generation module, which adopts a self-supervised monocular depth estimation network to receive the original monocular image, uses the absolute scale recovery technology, combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo laser point cloud generation module, which receives the normalized scale depth map, generates a pseudo laser point cloud through coordinate transformation and post-processing; a pseudo label generation module, which receives the pseudo laser point cloud, generates a pseudo label, and outputs a pseudo laser point cloud with a pseudo label; a multimodal fusion module, which receives the pseudo laser point cloud with a pseudo label and the original monocular image, and fuses them to obtain fusion features; a point cloud target detector, which realizes accurate positioning and classification of objects in three-dimensional space according to the fusion features.

[0038] The present invention has the following beneficial effects:

[0039] (1) The unsupervised domain adaptation strategy proposed in this paper for monocular 3D object detection effectively improves the performance and robustness of the system;

[0040] (2) Through the unsupervised learning mechanism, the present invention not only realizes accurate scale recovery in the depth estimation algorithm, but also achieves good results in the unsupervised domain adaptive target detection problem;

[0041] (3) The present invention uses a pseudo-label generation mechanism and a multimodal fusion target detector to fully utilize the original monocular image data and improve the stability of target detection;

[0042] (4) The present invention adopts an end-to-end joint training strategy to enhance the comprehensive detection capability of the model. The end-to-end pseudo laser point cloud algorithm can efficiently and accurately perform monocular three-dimensional target detection in the unsupervised domain adaptation scenario.

[0043] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A method step diagram of an embodiment of the present invention;

[0045] Figure 2 It is a monocular three-dimensional object detection network framework of an embodiment of the present invention;

[0046] Figure 3 is a schematic diagram of the training process of the random object scaling strategy in the pseudo label generation module of an embodiment of the present invention;

[0047] Figure 4 is a schematic diagram of feature weighted aggregation in an embodiment of the present invention;

[0048] Figure 5 is a schematic diagram of the training process of the point cloud detection model according to an embodiment of the present invention;

[0049] Figure 6 4 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0050] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0051] S101, constructing a monocular 3D target detection model and performing training;

[0052] S103, using the trained monocular 3D object detection model to implement monocular 3D object detection;

[0053] For details, see Figure 2As shown, the monocular 3D target detection model includes a depth map generation module, a pseudo laser point cloud generation module, a pseudo label generation module, a multimodal fusion module and a point cloud target detector. In the initial stage of the model, the depth map generation module can output a normalized scale depth map, that is, the depth information of the relative scale, through the self-supervised monocular depth estimation network. The absolute scale recovery technology is used to further restore the absolute depth value of each pixel point in combination with motion and geometric constraints. Then the pseudo laser point cloud generation module receives the depth information, forms a preliminary pseudo laser point cloud through the coordinate transformation process, and then applies the density-based interval sampling method to post-process the primary pseudo laser point cloud to simulate the output of the laser radar sensor and generate a pseudo laser point cloud. Before being used in the entire self-training process, the pseudo label generation model has been pre-trained in the source domain. The pre-training uses an object-based random scaling strategy to enhance the generalization performance of the pseudo label generation model. On this basis, the pseudo laser point cloud is input into the pseudo label generation module, and the model generates pseudo labels through the self-training process. Finally, the pseudo-labeled pseudo laser point cloud and the original monocular image sequence are input into the multimodal fusion point cloud target detection network (i.e., the multimodal fusion module and the point cloud target detector), thereby achieving accurate positioning and classification of objects in three-dimensional space. The entire process is completed within the model without manual intervention, and it is an end-to-end method. The model receives a monocular image sequence of continuous frames as input, and then outputs the result of pseudo point cloud three-dimensional target detection through end-to-end processing.

[0054] Specifically, pre-training refers to the use of labeled datasets (source domain data) to train the model in the early stage of building a monocular 3D object detection model, so that it can learn the basic features that can recognize and process 3D objects. This process helps the model capture key visual and depth information, laying the foundation for subsequent self-training. The pre-training process is for the entire 3D object detection model, including all components including the pseudo-label generation module. Point cloud data with real labels in the source domain are used to train and optimize the model to ensure that the model has good performance in the source domain, laying a solid foundation for subsequent self-training and unsupervised domain adaptation in the target domain. Self-training refers to the use of the knowledge learned by the model in the source domain after pre-training and applying it to the target domain (unlabeled dataset). In the self-training stage, the model generates pseudo laser point clouds and generates pseudo labels based on them, and then uses these pseudo labels to further train and optimize the model. This process enables the model to adapt to the data distribution of the target domain, improve the performance and robustness in the target domain, and achieve effective 3D object detection even in the absence of real labels in the target domain.

[0055] Specifically, the process of converting from a normalized scale depth map to a preliminary pseudo laser point cloud involves two key steps: coordinate transformation and post-processing. Through these two steps, a preliminary pseudo laser point cloud can be generated from the normalized scale depth map. This point cloud simulates the data structure and distribution of a real lidar and provides three-dimensional spatial information for subsequent target detection.

[0056] The purpose of coordinate transformation is to simulate the data acquisition process of LiDAR and convert the two-dimensional normalized scale depth map into three-dimensional point cloud data. This process includes the following sub-steps: Projection mapping: First, map each pixel in the normalized scale depth map into three-dimensional space. This usually involves converting the two-dimensional coordinates (x, y) and depth values ​​(d) of the image into three-dimensional coordinates (X, Y, Z). This conversion is based on the intrinsic parameters of the camera and the depth value of each pixel. Angle-distance grid construction: In three-dimensional space, the point cloud data is distributed according to the vertical angle (usually the pitch angle of the camera) to construct an angle-distance grid. This grid simulates the vertical angle distribution during LiDAR scanning, and each angle corresponds to a series of distance values. Scan line generation: Based on the angle-distance grid, a series of scan lines are generated. Each scan line consists of points with approximate vertical angles, which are arranged in order of distance values. In this way, each scan line simulates a scan of LiDAR at a specific angle.

[0057] The purpose of post-processing is to further optimize and clean the generated preliminary pseudo laser point cloud to improve the quality and usability of the point cloud data. Height range filtering: Remove points in the preliminary pseudo laser point cloud that are outside the specified height range. This height range is preset based on the characteristics of real lidar data and is used to remove outliers or irrelevant data points, such as points below the ground or points that are too high.

[0058] Specifically, Figure 3: is a training flow diagram of the random object scaling strategy in the pseudo label generation module of the present invention; this embodiment uses a pseudo label generation module to generate pseudo labels from pseudo laser point clouds, and uses it in a point cloud target detector of multimodal fusion. The core mechanism of this module lies in the random object scaling strategy. Before the step of generating pseudo labels in the target domain, it is necessary to pre-train the three-dimensional target detector using point cloud data with real labels in the source domain. The purpose of this step is to reduce the impact of the deviation brought by the source domain on the pseudo label generation effect, such as the deviation of object size and the deviation of target position. More specifically, the pre-trained model learns the basic discrimination method of target detection on the source data set, and then uses the learned weights to initialize the unlabeled data in the target domain. However, even if the manual annotation of the source domain is accurate enough, the pre-trained model may also generate learning deviations during the training process, such as the inconsistency between the target size and the point cloud density caused by the differences between different domains. Among these influences, the deviation of the target size directly affects the accuracy of three-dimensional target detection, resulting in errors in the size of the pseudo label bounding box generated in the target domain. The random object scaling strategy of the embodiment of the present invention can solve this problem.

[0059] Specifically, the object random scaling strategy is to simulate objects of different sizes in order to reduce the impact of object size deviation in the source domain data on the target domain. Specifically, for a labeled 3D object bounding box, which has a size (l,w,h), center coordinates (u,v,d) and orientation angle ω, first transform its points in the global coordinate system to the local coordinate system of the detection box:

[0060]

[0061] in, represents the coordinates of the i-th point in the detection box of size (l,w,h), and represents matrix multiplication. The process is to scale the length, width, and height of the detection box and transform the point from the global coordinate system to the local coordinate system of the detection box. Then, in order to get the scaled object, a random scaling factor (f l ,f w ,f h ) scales the point coordinates within the box to Finally, convert the scaled point coordinates back to the global point cloud coordinate system:

[0062]

[0063] Through the object random scaling strategy, objects of various sizes can be effectively simulated, solving the problem caused by size deviation between different domains in pre-training. This will help to generate more accurate point cloud pseudo-labels that are more robust to object size in the future.

[0064] Specifically, Figure 4 It is a schematic diagram of the multimodal fusion module of the present invention to realize feature weighted aggregation, and its core function is to establish richer information interaction between the features fused outside the candidate area and the features fused inside the candidate area; through this interaction, the model can better understand the contextual information of each area in the scene and how these areas are related to the overall scene. This helps the model to more accurately identify and locate objects when performing object detection, especially when dealing with occlusion and complex scenes. The process of realizing feature weighted aggregation includes the following steps: First, the feature weighted aggregation module receives multimodal features from the fusion outside the candidate area and the fusion inside the candidate area, including point cloud features and image features. Then, the self-attention mechanism is used to process these features. Self-attention allows the model to establish a weight relationship between features at different positions, so that the model can focus on the features most relevant to the current task. The weights output by the self-attention mechanism are used to weight the original features to generate weighted feature representations. This process allows the model to exchange information between features at different positions, strengthen key features and suppress irrelevant features. Finally, the weighted features are fused to form feature representations rich in interactive information, which will be used for subsequent point cloud target detection tasks. The fused features within the candidate region and the fused features outside the candidate region refer to the features of the potential target region and its background region predicted by the model, respectively. These features include point cloud and image information, which are weighted aggregated in the multimodal fusion module combined with the self-attention mechanism to form a comprehensive feature representation for improving the performance of the point cloud target detector; the original features refer to the basic attributes extracted directly from the input data without further processing, such as in monocular three-dimensional target detection, including visual features extracted from the original monocular image and spatial features of the point cloud obtained from the depth map.

[0065] For details, see Figure 5As shown, it is a schematic diagram of the training process of the multimodal unsupervised domain adaptive point cloud detection model described in the present invention. The model first uses the labeled source domain data to pre-train the point cloud detector through the source domain pre-training stage, so that the detector can capture key three-dimensional features. In the pre-training stage, in order to reduce the impact of the object size deviation in the source domain data on the target domain, the object random scaling strategy is used to simulate objects of different sizes, which helps the model learn generalized feature representations, which can be migrated to the target domain. Then, using the point cloud pseudo-label box, the model generates predicted bounding boxes without real labels. These pseudo-label boxes are based on the detection results of the pre-trained point cloud model and are further refined by the pseudo-label generation module. The pseudo-label module continuously updates and corrects itself during the self-training process to improve the accuracy of the pseudo-label. Subsequently, the multimodal fusion module combines the point cloud data with the original image sequence, and strengthens the feature representation through the candidate area fusion and candidate area fusion strategies, thereby improving the model's ability to recognize the target. Finally, the trained point cloud object detector can accurately predict the location and category of the target and output the point cloud detection results, achieving efficient domain adaptation performance even in the absence of labeled data in the target domain. This process not only reduces the dependence on labeled data, but also significantly improves the generalization ability and detection accuracy of the model in different environments and application scenarios.

[0066] The specific process of S102 is: first input a monocular image sequence, obtain a depth map through self-supervised monocular depth estimation, generate a pseudo point cloud, then generate a point cloud pseudo label, and finally perform a multimodal fusion point cloud target detector to achieve end-to-end monocular three-dimensional target detection. It should be noted that in traditional point cloud quantization, the quantization process is discrete. Its advantage is that features can be directly extracted from tensors using a simple binary representation, but this makes back propagation very difficult. To address this problem, the present invention introduces a soft quantization method, which enables the network to calculate gradients and back propagate during the quantization process, so that the entire pseudo laser point cloud model can be trained end-to-end. Soft quantization does not simply use a binary form to represent the occupancy tensor, but uses a radial basis function weighted method to perform calculations. Specifically, given a set of point cloud data P = {p1, p2, p3, ..., p N}, the goal is to generate a 3D occupancy tensor T softquantization , where each unit u is associated with a fixed center position p c If point p falls outside unit u, it occupies tensor T soft quantization (u) = 0; if point p falls within unit u, then:

[0067]

[0068] Among them, W(u) is the radial basis function weight, δ represents a very small positive value, represents the neighboring unit in the neighborhood u of unit u. There is a special case here when δ is much larger than 0 and r u When is an empty set, the above soft quantization will be restored to the original binary definition. Through this representation change method, the point cloud occupancy tensor can be differentiated, and then the depth estimation network and the point cloud detection network can be jointly optimized through back propagation. Through this point cloud soft quantization method, the entire monocular pseudo-laser point cloud model can be trained end-to-end to complete the entire monocular 3D target detection task.

[0069] The embodiment of the present invention further provides an unsupervised domain adaptive monocular 3D target detection system, comprising:

[0070] Model training module 601, constructing a monocular 3D object detection model and performing training;

[0071] The object detection module 602 implements monocular 3D object detection using the trained monocular 3D object detection model.

[0072] The present invention provides an unsupervised domain adaptive monocular 3D target detection method, and proposes a pseudo laser point cloud generation method, a point cloud pseudo label generation method and an end-to-end unsupervised domain adaptive monocular 3D target detection framework. The output of 3D target detection results can be achieved by directly inputting a continuous monocular image sequence with high accuracy and robustness. The embodiment of the present invention can run in real time on a single-card GPU, and running monocular 3D target detection on an RTX 3080 alone can reach 31fps.

[0073] Although the present invention is disclosed as above, it is not limited thereto. For those skilled in the art, the technical solutions and inventive concepts of the present invention can be replaced and changed without departing from the spirit and scope of the present invention, and the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. An unsupervised domain adaptive monocular 3D object detection method, characterized in that: The following steps are involved: Build a monocular 3D target detection model and perform training; Use the trained monocular 3D target detection model to realize monocular 3D target detection; The monocular 3D target detection model includes: a depth map generation module, which adopts a self-supervised monocular depth estimation network to receive the original monocular image, uses the absolute scale recovery technology, combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo laser point cloud generation module, which receives the normalized scale depth map, generates a pseudo laser point cloud through coordinate transformation and post-processing; a pseudo label generation module, which receives the pseudo laser point cloud, generates a pseudo label, and outputs a pseudo laser point cloud with a pseudo label; a multimodal fusion module, which receives the pseudo laser point cloud with a pseudo label and the original monocular image, and fuses them to obtain fusion features; a point cloud target detector, which realizes accurate positioning and classification of objects in three-dimensional space according to the fusion features.

2. The unsupervised domain adaptive monocular 3D object detection method according to claim 1, characterized in that: The training includes: Pre-training: In the early stages of building a monocular 3D object detection model, a labeled dataset is used as source domain data to train the model so that it can learn the basic features that can recognize and process 3D objects, helping the model capture key visual and depth information. Self-training: The pre-trained monocular 3D object detection model is applied to an unlabeled dataset, i.e., target domain data. The model generates a pseudo laser point cloud and generates pseudo labels based on it, and then uses these pseudo labels to further train and optimize the model.

3. The unsupervised domain adaptive monocular 3D object detection method according to claim 2, characterized in that: The pre-training includes pre-training of a pseudo-label generation model and pre-training of a point cloud target detector. The pre-training process adopts an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model and improve the domain adaptation performance of the point cloud target detector.

4. The unsupervised domain adaptive monocular 3D object detection method according to claim 3, characterized in that: The object random scaling strategy includes the following steps: For a labeled 3D object bounding box, with size (l, w, h), center coordinates (u, v, d) and orientation angle ω, scaling transformation is performed along the length, width and height of the detection box to convert the point from the global coordinate system to the local coordinate system of the detection box, expressed as: in, represents the coordinates of the i-th point in the detection box of size (l,w,h), represents matrix multiplication,; By random scaling factor (f l ,f w ,f h ) scales the point coordinates within the box to Convert the scaled point coordinates back to the global point cloud coordinate system:

5. The unsupervised domain adaptive monocular 3D object detection method according to claim 2, characterized in that: The self-training includes the self-training of the pseudo-label generation module, which guides the generation process of the target domain pseudo-label by introducing the quality-aware three-interval memory library, including the following steps: Get the prediction box generated by the point cloud detector; The quality of the predicted box is evaluated based on the IoU scoring criterion, which is pre-trained in the source domain based on the object random scaling strategy; A three-interval prediction box partitioning scheme is used to guide the memory library update and pseudo-label generation. The memory library is updated based on the fusion of the current pseudo-label and the historical pseudo-label, and the pseudo-label required for subsequent training is continuously updated.

6. The unsupervised domain adaptive monocular 3D object detection method according to claim 1, characterized in that: The process of generating a pseudo laser point cloud by the pseudo laser point cloud generating module comprises the following steps: The data acquisition process of the laser radar is simulated through coordinate transformation; the projection of the point cloud on the vertical plane is mapped and distributed according to the vertical angle to construct an angle-distance grid; based on the angle-distance grid, a series of scan lines are generated to simulate the scanning of the laser radar and generate a preliminary laser point cloud; each scan line is composed of points with an approximate vertical angle and arranged in order of distance value, and the point closest to the radar is regarded as the first obstacle encountered by the corresponding laser beam; The preliminary pseudo laser point cloud is post-processed; pseudo laser radar data points located outside a specified height range in the preliminary pseudo laser point cloud are removed, and the specified height range is preset according to the characteristics of the real laser radar data.

7. The unsupervised domain adaptive monocular 3D object detection method according to claim 1, characterized in that: The multimodal fusion module uses a feature weighted aggregation mechanism to fuse point cloud data with the original monocular image, including the following steps: Receiving fused features from outside the candidate region and fused features from within the candidate region, including point cloud features and image features; Use the self-attention mechanism to establish a weight relationship between features at different positions, output weights to weight the original features, and generate weighted features; The weighted features are fused to form a feature representation containing interactive information, which is input into the point cloud object detector as the fused features.

8. The unsupervised domain adaptive monocular 3D object detection method according to claim 1, characterized in that: Before the multimodal fusion module fuses the pseudo-labeled laser point cloud with the original monocular image, the pseudo-labeled laser point cloud is quantized using a soft quantization method: a radial basis function weighted method is used for calculation; for a given set of point cloud data P = {p1, p2, p3, ..., p N }, a 3D occupancy tensor T is generated using a soft quantization method softquantization , where each unit u is associated with a fixed center position p c If point p falls outside unit u, T softquantization (u) = 0; if point p falls within unit u, it is expressed as: Among them, W(u) is the radial basis function weight, δ represents a very small positive value, represents the neighboring unit in the neighborhood u of unit u; when δ is much larger than 0 and the neighborhood r of unit u u When is an empty set, the above soft quantization returns to the original binary definition.

9. An unsupervised domain adaptive monocular 3D object detection system, characterized in that: include: Model training module, builds a monocular 3D target detection model and performs training; The object detection module uses the trained monocular 3D object detection model to realize monocular 3D object detection; The monocular 3D target detection model includes: a depth map generation module, which adopts a self-supervised monocular depth estimation network to receive the original monocular image, uses the absolute scale recovery technology, combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo laser point cloud generation module, which receives the normalized scale depth map, generates a pseudo laser point cloud through coordinate transformation and post-processing; a pseudo label generation module, which receives the pseudo laser point cloud, generates a pseudo label, and outputs a pseudo laser point cloud with a pseudo label; a multimodal fusion module, which receives the pseudo laser point cloud with a pseudo label and the original monocular image, and fuses them to obtain fusion features; a point cloud target detector, which realizes accurate positioning and classification of objects in three-dimensional space according to the fusion features.

Images