An unsupervised domain adaptive monocular 3D object detection method

Abstract

This invention discloses an unsupervised domain adaptive monocular 3D target detection method and system. The method includes: constructing a monocular 3D target detection model and performing pre-training and self-training; using the trained monocular 3D target detection model to perform monocular 3D target detection; the monocular 3D target detection model includes: a depth map generation module, which receives the original monocular image and outputs a normalized scale depth map containing relative scale depth information; a pseudo-laser point cloud generation module, which receives the normalized scale depth map and generates pseudo-laser point clouds; a pseudo-label generation module, which receives the pseudo-laser point clouds and outputs pseudo-laser point clouds with pseudo-labels; a multimodal fusion module, which receives the pseudo-laser point clouds with pseudo-labels and the original monocular image, and fuses them to obtain fusion features; and a point cloud target detector, which realizes accurate localization and classification of objects in 3D space based on the fusion features. The unsupervised domain adaptive strategy of this invention effectively improves the performance and robustness of the system.

Description

Technical Field

[0001] This invention relates to the fields of image processing and deep learning, and in particular to an unsupervised domain adaptive monocular 3D target detection method. Background Technology

[0002] 3D object detection is widely used in environmental perception, scene understanding, and 3D reconstruction, playing a crucial role in these fields. Vision-based 3D object detection sensors are divided into monocular and binocular sensors. Compared to binocular 3D object detection, monocular 3D object detection has advantages such as lower cost, simpler installation, and easier data acquisition, making it a more challenging and promising research direction. Most current monocular 3D object detection methods are based on supervised learning, which requires acquiring a large number of accurate ground truth labels, a significant challenge. Therefore, unsupervised adaptive monocular 3D object detection models are closer to practical applications. However, unsupervised adaptive methods face the problem of significant performance degradation in cross-domain applications.

[0003] Meanwhile, the pseudo-laser point cloud method is an important approach in monocular 3D target detection. It fully utilizes the depth information obtained from monocular depth estimation to construct a pseudo-laser point cloud, and then uses a laser point cloud detector to complete target detection. However, this usually requires two stages, and the problem is that it cannot fully utilize the correlation between depth estimation and target detection, leading to significant errors. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art.

[0005] The technical solution adopted by this invention to solve its technical problem is: to provide an unsupervised domain adaptive monocular 3D target detection method, comprising the following steps:

[0006] Construct and train a monocular 3D target detection model;

[0007] Monocular 3D target detection is achieved using a trained monocular 3D target detection model.

[0008] The monocular 3D target detection model includes: a depth map generation module, which uses a self-supervised monocular depth estimation network to receive the original monocular image, uses absolute scale recovery technology, and combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo-laser point cloud generation module, which receives the normalized scale depth map and generates pseudo-laser point clouds through coordinate transformation and post-processing; a pseudo-label generation module, which receives the pseudo-laser point clouds, generates pseudo-labels, and outputs pseudo-laser point clouds with pseudo-labels; a multimodal fusion module, which receives the pseudo-laser point clouds with pseudo-labels and the original monocular image, and fuses them to obtain fusion features; and a point cloud target detector, which realizes accurate localization and classification of objects in 3D space based on the fusion features.

[0009] Preferably, the training includes:

[0010] Pre-training involves using labeled datasets as source domain data to train the model in the early stages of building a monocular 3D object detection model. This allows the model to learn the basic features that enable it to recognize and process 3D objects, helping it capture key visual and depth information.

[0011] Self-training involves applying the pre-trained monocular 3D object detection model to an unlabeled dataset, i.e., object domain data. The model generates pseudo-laser point clouds and pseudo-labels based on them, and then uses these pseudo-labels to further train and optimize the model.

[0012] Preferably, the pre-training includes pre-training the pseudo-label generation model and pre-training the point cloud object detector. The pre-training process adopts an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model and improve the domain adaptation performance of the point cloud object detector.

[0013] Preferably, the object random scaling strategy includes the following steps:

[0014] For a labeled 3D target bounding box with size (l, w, h), center coordinates (u, v, d), and orientation angle ω, scaling the points along the length, width, and height of the bounding box transforms the points from the global coordinate system to the local coordinate system of the bounding box, as follows:

[0015]

[0016] in, Let represent the coordinates of the i-th point within a detection box of size (l, w, h), and · represent matrix multiplication.

[0017] By random scaling factor (f) l ,f w ,f h Scale the coordinates of the points within the box to

[0018] Convert the scaled point coordinates back to the global point cloud coordinate system:

[0019]

[0020] Preferably, the self-training includes the self-training of the pseudo-label generation module, which guides the generation process of target domain pseudo-labels by introducing a quality-aware three-interval memory library, including the following steps:

[0021] Obtain the predicted bounding boxes generated by the point cloud detector;

[0022] The quality of the predicted bounding boxes is evaluated based on the IoU-based scoring criterion, which is based on pre-training in the source domain using a random scaling strategy for objects.

[0023] A three-interval prediction box partitioning scheme is adopted to guide the updating of the memory library and the generation of pseudo-labels. The memory library is updated based on the fusion of the currently generated pseudo-labels and historical pseudo-labels, and the pseudo-labels required for subsequent training are continuously updated.

[0024] Preferably, the process of generating a pseudo-laser point cloud by the pseudo-laser point cloud generation module includes the following steps:

[0025] The data acquisition process of lidar is simulated by coordinate transformation; the projection of the point cloud on the vertical plane is mapped and distributed according to the vertical angle to construct an angle-distance grid; based on the angle-distance grid, a series of scan lines are generated to simulate the scanning of lidar and generate a preliminary laser point cloud; each scan line is composed of points with an approximately vertical angle and arranged in order of distance value, and the point closest to the radar is regarded as the first obstacle encountered by the corresponding laser beam;

[0026] Post-processing is performed on the initial pseudo-laser point cloud; pseudo-laser radar data points located outside the specified height range in the initial pseudo-laser point cloud are removed. The specified height range is preset based on the characteristics of real laser radar data.

[0027] Preferably, the multimodal fusion module uses a feature-weighted aggregation mechanism to fuse point cloud data with the original monocular image, including the following steps:

[0028] Receive fusion features from outside the candidate region and fusion features from within the candidate region, including point cloud features and image features;

[0029] By using the self-attention mechanism, a weight relationship is established between features at different locations, and the weights are output to weight the original features, generating weighted features.

[0030] The weighted features are fused to form a feature representation containing interactive information, which is then used as the input of the fused features to the point cloud target detector.

[0031] Preferably, before fusing the pseudo-labeled laser point cloud and the original monocular image, the multimodal fusion module performs point cloud quantization on the pseudo-labeled laser point cloud using a soft quantization method: a radial basis function weighting method is used for calculation; for a given set of point cloud data P = {p1, p2, p3, ..., pN}, a 3D occupancy tensor T is generated using the soft quantization method. softquantization Each unit u is associated with a fixed center position p. c Association, if point p falls outside cell u, T soft quantization (u) = 0; if point p falls within cell u, then it is represented as:

[0032]

[0033] Where W(u) is the radial basis function weight, and δ represents a very small positive value. Represents the neighboring cells in the neighborhood of cell u; when δ is much greater than 0 and the neighborhood r of cell u is greater than 0. u When the set is empty, the soft quantization described above reverts to the original binary definition.

[0034] The present invention also provides an unsupervised domain adaptive monocular 3D target detection system, comprising:

[0035] The model training module constructs and trains a monocular 3D object detection model.

[0036] The target detection module utilizes a trained monocular 3D target detection model to achieve monocular 3D target detection.

[0037] The monocular 3D target detection model includes: a depth map generation module, which uses a self-supervised monocular depth estimation network to receive the original monocular image, uses absolute scale recovery technology, and combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo-laser point cloud generation module, which receives the normalized scale depth map and generates pseudo-laser point clouds through coordinate transformation and post-processing; a pseudo-label generation module, which receives the pseudo-laser point clouds, generates pseudo-labels, and outputs pseudo-laser point clouds with pseudo-labels; a multimodal fusion module, which receives the pseudo-laser point clouds with pseudo-labels and the original monocular image, and fuses them to obtain fusion features; and a point cloud target detector, which realizes accurate localization and classification of objects in 3D space based on the fusion features.

[0038] The present invention has the following beneficial effects:

[0039] (1) The unsupervised domain adaptive strategy proposed in this invention for monocular 3D target detection effectively improves the performance and robustness of the system;

[0040] (2) Through the unsupervised learning mechanism, this invention not only achieves accurate scale recovery in depth estimation algorithms, but also achieves good results in the problem of adaptive target detection in unsupervised domains;

[0041] (3) This invention simultaneously utilizes a pseudo-label generation mechanism and a multimodal fusion target detector to make full use of the original monocular image data and improve the stability of target detection;

[0042] (4) The present invention adopts an end-to-end joint training strategy to enhance the comprehensive detection capability of the model. The end-to-end pseudo laser point cloud algorithm can perform monocular three-dimensional target detection efficiently and accurately in unsupervised adaptive scenarios.

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. Attached Figure Description

[0044] Figure 1 This is a diagram illustrating the method steps of an embodiment of the present invention;

[0045] Figure 2 This is a monocular 3D target detection network framework according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the training process of the object random scaling strategy in the pseudo-label generation module of this invention.

[0047] Figure 4 This is a schematic diagram of feature weighted aggregation according to an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of the training process of the point cloud detection model according to an embodiment of the present invention;

[0049] Figure 6 This is a system structure diagram of an embodiment of the present invention. Detailed Implementation

[0050] See Figure 1 The diagram shows the method steps of an embodiment of the present invention, including the following steps:

[0051] S101, Construct a monocular 3D target detection model and train it;

[0052] S103, using a trained monocular 3D target detection model to achieve monocular 3D target detection;

[0053] For details, see Figure 2As shown, the monocular 3D target detection model includes a depth map generation module, a pseudo-laser point cloud generation module, a pseudo-label generation module, a multimodal fusion module, and a point cloud target detector. In the initial stage of the model, the depth map generation module, through a self-supervised monocular depth estimation network, outputs a depth map at a normalized scale, i.e., depth information at a relative scale. Using absolute scale recovery technology, combined with motion and geometric constraints, the absolute depth value of each pixel is further recovered. Next, the pseudo-laser point cloud generation module receives this depth information, forms a preliminary pseudo-laser point cloud through a coordinate transformation process, and then applies a density-based interval sampling method to post-process the preliminary pseudo-laser point cloud, thereby simulating the output of a lidar sensor to generate a pseudo-laser point cloud. The pseudo-label generation model has been pre-trained in the source domain before being used in the entire self-training process. Pre-training uses an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model. Based on this, the pseudo-laser point cloud is input into the pseudo-label generation module, and the model generates pseudo-labels through the self-training process. Finally, the pseudo-labeled pseudo-laser point cloud and the original monocular image sequence are input into a multimodal fusion point cloud target detection network (i.e., a multimodal fusion module and a point cloud target detector), thereby achieving accurate localization and classification of objects in 3D space. The entire process is completed internally within the model, requiring no manual intervention, and is an end-to-end method. The model receives a continuous sequence of monocular images as input, and then, through end-to-end processing, outputs the results of pseudo-point cloud 3D target detection.

[0054] Specifically, pre-training refers to the initial stage of building a monocular 3D object detection model. It involves training the model using a labeled dataset (source domain data) to teach it the basic features needed to recognize and process 3D objects. This process helps the model capture crucial visual and depth information, laying the foundation for subsequent self-training. Pre-training applies to the entire 3D object detection model, including all components such as the pseudo-label generation module. It uses point cloud data with real labels from the source domain to train and optimize the model, ensuring good performance in the source domain and providing a solid foundation for subsequent self-training in the target domain and unsupervised domain adaptation. Self-training, following pre-training, involves applying the knowledge learned in the source domain to the target domain (unlabeled dataset). During self-training, the model generates pseudo-laser point clouds and pseudo-labels based on these, then uses these pseudo-labels to further train and optimize the model. This process enables the model to adapt to the data distribution of the target domain, improving its performance and robustness in that domain, achieving effective 3D object detection even when real labels are lacking.

[0055] Specifically, the process of converting a normalized scale depth map into a preliminary pseudo-laser point cloud involves two key steps: coordinate transformation and post-processing. These two steps generate a preliminary pseudo-laser point cloud from the normalized scale depth map. This point cloud simulates the data structure and distribution of a real lidar, providing three-dimensional spatial information for subsequent target detection.

[0056] The purpose of coordinate transformation is to simulate the data acquisition process of LiDAR, converting a two-dimensional normalized depth map into three-dimensional point cloud data. This process includes the following sub-steps: Projection Mapping: First, each pixel in the normalized depth map is mapped to three-dimensional space. This typically involves converting the image's two-dimensional coordinates (x, y) and depth values ​​(d) into three-dimensional coordinates (X, Y, Z). This transformation is based on the camera's intrinsic parameters and the depth value of each pixel. Angle-Distance Grid Construction: In three-dimensional space, the point cloud data is distributed according to vertical angles (usually the camera's pitch angle) to construct an angle-distance grid. This grid simulates the vertical angle distribution during LiDAR scanning, with each angle corresponding to a series of distance values. Scan Line Generation: Based on the angle-distance grid, a series of scan lines are generated. Each scan line consists of points with approximately vertical angles, arranged in order of distance values. In this way, each scan line simulates a scan by the LiDAR at a specific angle.

[0057] The purpose of post-processing is to further optimize and clean the generated preliminary pseudo-LiDAR point cloud to improve the quality and usability of the point cloud data. Height range filtering: Removes points in the preliminary pseudo-LiDAR point cloud that are outside a specified height range. This height range is preset based on the characteristics of real LiDAR data and is used to remove outliers or irrelevant data points, such as points below ground level or excessively high points.

[0058] Specifically, Figure 3This is a schematic diagram of the training process of the object random scaling strategy in the pseudo-label generation module of the present invention. This embodiment utilizes a proposed pseudo-label generation module to generate pseudo-labels from pseudo-laser point clouds and applies it to a multimodal fusion point cloud target detector. The core mechanism of this module lies in the object random scaling strategy. Before generating pseudo-labels in the target domain, the 3D target detector needs to be pre-trained on point cloud data with real labels in the source domain. The purpose of this step is to mitigate the impact of biases from the source domain on the pseudo-label generation effect, such as biases in object size and target position. More specifically, the pre-trained model learns the basic discrimination method for target detection on the source dataset, and then uses the learned weights to initialize the unlabeled data in the target domain. However, even if the manual annotations in the source domain are accurate enough, the pre-trained model may still generate learning biases during training, such as inconsistencies between target size and point cloud density due to differences between different domains. Among these influences, the deviation in target size directly affects the accuracy of 3D target detection, resulting in errors in the bounding box size of the pseudo-labels generated in the target domain. The object random scaling strategy of this embodiment can solve this problem.

[0059] Specifically, the object random scaling strategy simulates objects of different sizes to reduce the impact of object size deviations in the source domain data on the target domain. Specifically, for a labeled 3D target bounding box with size (l, w, h), center coordinates (u, v, d), and orientation angle ω, the points in its global coordinate system are first transformed to the local coordinate system of the detection box:

[0060]

[0061] in, This represents the coordinates of the i-th point within a detection box of size (l, w, h), where · represents matrix multiplication. This process involves scaling the points along the length, width, and height of the detection box, transforming them from the global coordinate system to the local coordinate system of the detection box. Then, to obtain the scaled object, a random scaling factor (f) is used. l ,f w ,f h Scale the coordinates of the points within the box to Finally, the scaled point coordinates are transformed back to the global point cloud coordinate system:

[0062]

[0063] By employing a random scaling strategy for objects, objects of various sizes can be effectively simulated, resolving issues caused by size discrepancies between different domains during pre-training. This will facilitate the subsequent generation of more robust and accurate point cloud pseudo-labels that are more robust to object size.

[0064] Specifically, Figure 4 This diagram illustrates the feature weighted aggregation implemented by the multimodal fusion module of this invention. Its core function is to establish richer information interaction between features fused outside and within candidate regions. Through this interaction, the model can better understand the contextual information of each region in the scene and how these regions relate to the overall scene. This helps the model more accurately identify and locate objects during object detection, especially when dealing with occlusion and complex scenes. The process of implementing feature weighted aggregation includes the following steps: First, the feature weighted aggregation module receives multimodal features from both outside and within candidate regions, including point cloud features and image features. Next, a self-attention mechanism is used to process these features. Self-attention allows the model to establish weight relationships between features at different locations, allowing the model to focus on features most relevant to the current task. The weights output by the self-attention mechanism are used to weight the original features, thereby generating a weighted feature representation. This process allows the model to exchange information between features at different locations, strengthening key features and suppressing irrelevant features. Finally, the weighted features are fused to form a feature representation rich in interactive information, which will be used for subsequent point cloud object detection tasks. The fusion features within the candidate region and the fusion features outside the candidate region refer to the features of the potential target region and its background region predicted by the model, respectively. These features include point cloud and image information. They are weighted and aggregated in the multimodal fusion module using a self-attention mechanism to form a comprehensive feature representation, which is used to improve the performance of the point cloud target detector. The original features refer to the basic attributes extracted directly from the input data without further processing. For example, in monocular 3D target detection, these include visual features extracted from the original monocular image and spatial features of the point cloud obtained from the depth map.

[0065] For details, see Figure 5The diagram illustrates the training process of the multimodal unsupervised adaptive point cloud detection model described in this invention. First, the model undergoes a source domain pre-training phase, where the point cloud detector is pre-trained using labeled source domain data, enabling it to capture key 3D features. During this phase, to reduce the impact of object size deviations in the source domain data on the target domain, a random scaling strategy is used to simulate objects of different sizes, helping the model learn generalized feature representations that can be transferred to the target domain. Next, using pseudo-labeled bounding boxes, the model generates predicted bounding boxes without real labels. These pseudo-labeled boxes are based on the detection results of the pre-trained point cloud model and are further refined by the pseudo-label generation module. The pseudo-label module continuously updates and corrects itself during self-training, improving the accuracy of the pseudo-labels. Finally, the multimodal fusion module combines the point cloud data with the original image sequence, using fusion strategies within and outside candidate regions to strengthen feature representations and improve the model's ability to recognize targets. Ultimately, the trained point cloud object detector can accurately predict the location and category of objects and output point cloud detection results, achieving efficient domain adaptation performance even in situations where labeled data is lacking in the target domain. This process not only reduces reliance on labeled data but also significantly improves the model's generalization ability and detection accuracy in different environments and application scenarios.

[0066] The specific process of S102 is as follows: First, a monocular image sequence is input, a depth map is obtained through self-supervised monocular depth estimation, a pseudo-point cloud is generated, then point cloud pseudo-labels are generated, and finally a multimodal fusion point cloud target detector is performed to achieve end-to-end monocular 3D target detection. It should be noted that in traditional point cloud quantization, the quantization process is discrete. Its advantage is that features can be directly extracted from the tensor using a simple binary representation, but this makes backpropagation very difficult. To address this problem, this invention introduces a soft quantization method, enabling the network to calculate gradients and backpropagation during the quantization process, thus allowing the entire pseudo-laser point cloud model to be trained end-to-end. Soft quantization does not simply use a binary representation of the occupancy tensor, but uses a radial basis function weighted method for calculation. Specifically, given a set of point cloud data P = {p1, p2, p3, ..., p...} N The goal is to generate a 3D occupancy tensor T. softquantization Each unit u is associated with a fixed center position p. c Association. If point p falls outside cell u, then it occupies tensor T. soft quantization (u) = 0; if point p falls within element u, then:

[0067]

[0068] Where W(u) is the radial basis function weight, and δ represents a very small positive value. This represents the neighboring cells in the neighborhood of cell u. There is a special case where δ is much greater than 0 and r... u When the set is empty, the soft quantization described above will revert to the original binarization definition. This representation transformation method allows for differentiation of the point cloud occupancy tensor, which can then be used to jointly optimize the depth estimation network and the point cloud detection network via backpropagation. Through this point cloud soft quantization method, the entire monocular pseudo-laser point cloud model can achieve end-to-end training, completing the entire monocular 3D target detection task.

[0069] This invention also provides an unsupervised domain adaptive monocular 3D target detection system, comprising:

[0070] Model training module 601 constructs and trains a monocular 3D object detection model;

[0071] The target detection module 602 uses a trained monocular 3D target detection model to achieve monocular 3D target detection.

[0072] This invention provides an unsupervised adaptive monocular 3D target detection method, proposing a pseudo-laser point cloud generation method, a point cloud pseudo-label generation method, and an end-to-end unsupervised adaptive monocular 3D target detection framework. By directly inputting a continuous sequence of monocular images, the method can output 3D target detection results with high accuracy and robustness. Embodiments of this invention can run in real-time on a single GPU; monocular 3D target detection on a standalone RTX 3080 can achieve 31fps.

[0073] While the present invention has been disclosed above, it is not limited thereto. Those skilled in the art will be able to make substitutions and modifications based on the technical solutions and inventive concepts of the present invention without departing from the spirit and scope thereof, and the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An unsupervised, domain-adaptive monocular 3D target detection method, characterized in that, Includes the following steps: Construct and train a monocular 3D target detection model; Monocular 3D target detection is achieved using a trained monocular 3D target detection model. The monocular 3D target detection model includes: a depth map generation module, which uses a self-supervised monocular depth estimation network to receive the original monocular image, uses absolute scale recovery technology, and combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo-laser point cloud generation module, which receives the normalized scale depth map and generates a pseudo-laser point cloud through coordinate transformation and post-processing; a pseudo-label generation module, which receives the pseudo-laser point cloud, generates pseudo-labels, and outputs a pseudo-laser point cloud with pseudo-labels; a multimodal fusion module, which receives the pseudo-laser point cloud with pseudo-labels and the original monocular image, and fuses them to obtain fusion features; and a point cloud target detector, which realizes accurate localization and classification of objects in 3D space based on the fusion features. The training includes: Pre-training involves using labeled datasets as source domain data to train the model in the early stages of building a monocular 3D object detection model. This allows the model to learn the basic features that enable it to recognize and process 3D objects, helping it capture key visual and depth information. Self-training involves applying the pre-trained monocular 3D target detection model to an unlabeled dataset, i.e., target domain data. The model generates pseudo-laser point clouds and pseudo-labels based on them, and then uses these pseudo-labels to further train and optimize the model. The pre-training includes pre-training the pseudo-label generation model and pre-training the point cloud object detector. The pre-training process adopts an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model and improve the domain adaptation performance of the point cloud object detector. The random scaling strategy for the object includes the following steps: For a labeled 3D target bounding box with size (l, w, h), center coordinates (u, v, d), and orientation angle ω, scaling the points along the length, width, and height of the bounding box transforms the points from the global coordinate system to the local coordinate system of the bounding box, as follows: in, This represents the coordinates of the i-th point within a detection box of size (l, w, h), where · represents matrix multiplication; By random scaling factor (f) l ,f w ,f h Scale the coordinates of the points within the box to Convert the scaled point coordinates back to the global point cloud coordinate system: Before fusing the pseudo-labeled laser point cloud with the original monocular image, the multimodal fusion module performs point cloud quantization using a soft quantization method: a radial basis function weighting method is used for calculation; for a given set of point cloud data P = {p1, p2, p3, ..., p N A 3D occupancy tensor T is generated using a soft quantization method. softquantization Each unit u is associated with a fixed center position p. c Association, if point p falls outside cell u, T softquantization (u) = 0; if point p falls within cell u, then it is represented as: Where W(u) is the radial basis function weight, and δ represents a very small positive value. Represents the neighboring cells in the neighborhood of cell u; when δ is much greater than 0 and the neighborhood r of cell u is greater than 0, the neighboring cells in the neighborhood of cell u are greater than 0. u When the set is empty, the soft quantization described above reverts to the original binary definition.

2. The unsupervised domain adaptive monocular 3D target detection method according to claim 1, characterized in that, The self-training includes the self-training of the pseudo-label generation module, which guides the generation process of target domain pseudo-labels by introducing a quality-aware three-interval memory library, including the following steps: Obtain the predicted bounding boxes generated by the point cloud detector; The IoU-based scoring criterion evaluates the quality of the predicted bounding boxes and is based on pre-training in the source domain using a random scaling strategy for objects. A three-interval prediction box partitioning scheme is adopted to guide the updating of the memory library and the generation of pseudo-labels. The memory library is updated based on the fusion of the currently generated pseudo-labels and historical pseudo-labels, and the pseudo-labels required for subsequent training are continuously updated.

3. The unsupervised domain adaptive monocular 3D target detection method according to claim 1, characterized in that, The process of generating a pseudo laser point cloud by the pseudo laser point cloud generation module includes the following steps: The data acquisition process of lidar is simulated by coordinate transformation; the projection of the point cloud on the vertical plane is mapped and distributed according to the vertical angle to construct an angle-distance grid; based on the angle-distance grid, a series of scan lines are generated to simulate the scanning of lidar and generate a preliminary laser point cloud; each scan line is composed of points with an approximately vertical angle and arranged in order of distance value, and the point closest to the radar is regarded as the first obstacle encountered by the corresponding laser beam; Post-processing is performed on the initial pseudo-laser point cloud; pseudo-laser radar data points located outside the specified height range in the initial pseudo-laser point cloud are removed. The specified height range is preset based on the characteristics of real laser radar data.

4. The unsupervised domain adaptive monocular 3D target detection method according to claim 1, characterized in that, The multimodal fusion module uses a feature-weighted aggregation mechanism to fuse point cloud data with the original monocular image, including the following steps: Receive fusion features from outside the candidate region and fusion features from within the candidate region, including point cloud features and image features; By using the self-attention mechanism, a weight relationship is established between features at different locations, and the weights are output to weight the original features, generating weighted features. The weighted features are fused to form a feature representation containing interactive information, which is then used as the input of the fused features to the point cloud target detector.

5. An unsupervised domain adaptive monocular three-dimensional target detection system, characterized in that, include: The model training module constructs and trains a monocular 3D object detection model. The target detection module utilizes a trained monocular 3D target detection model to achieve monocular 3D target detection. The monocular 3D target detection model includes: a depth map generation module, which uses a self-supervised monocular depth estimation network to receive the original monocular image, uses absolute scale recovery technology, and combines motion and geometric constraints to recover the absolute depth value of each pixel in the original monocular image, and outputs a normalized scale depth map containing relative scale depth information; a pseudo-laser point cloud generation module, which receives the normalized scale depth map and generates a pseudo-laser point cloud through coordinate transformation and post-processing; a pseudo-label generation module, which receives the pseudo-laser point cloud, generates pseudo-labels, and outputs a pseudo-laser point cloud with pseudo-labels; a multimodal fusion module, which receives the pseudo-laser point cloud with pseudo-labels and the original monocular image, and fuses them to obtain fusion features; and a point cloud target detector, which realizes accurate localization and classification of objects in 3D space based on the fusion features. The training includes: Pre-training involves using labeled datasets as source domain data to train the model in the early stages of building a monocular 3D object detection model. This allows the model to learn the basic features that enable it to recognize and process 3D objects, helping it capture key visual and depth information. Self-training involves applying the pre-trained monocular 3D target detection model to an unlabeled dataset, i.e., target domain data. The model generates pseudo-laser point clouds and pseudo-labels based on them, and then uses these pseudo-labels to further train and optimize the model. The pre-training includes pre-training the pseudo-label generation model and pre-training the point cloud object detector. The pre-training process adopts an object-based random scaling strategy to enhance the generalization performance of the pseudo-label generation model and improve the domain adaptation performance of the point cloud object detector. The random scaling strategy for the object includes the following steps: For a labeled 3D target bounding box with size (l, w, h), center coordinates (u, v, d), and orientation angle ω, scaling the points along the length, width, and height of the bounding box transforms the points from the global coordinate system to the local coordinate system of the bounding box, as follows: in, This represents the coordinates of the i-th point within a detection box of size (l, w, h), where · represents matrix multiplication; By random scaling factor (f) l ,f w ,f h Scale the coordinates of the points within the box to Convert the scaled point coordinates back to the global point cloud coordinate system: Before fusing the pseudo-labeled laser point cloud with the original monocular image, the multimodal fusion module performs point cloud quantization using a soft quantization method: a radial basis function weighting method is used for calculation; for a given set of point cloud data P = {p1, p2, p3, ..., p N A 3D occupancy tensor T is generated using a soft quantization method. softquantization Each unit u is associated with a fixed center position p. c Association, if point p falls outside cell u, T softquantization (u) = 0; if point p falls within cell u, then it is represented as: Where W(u) is the radial basis function weight, and δ represents a very small positive value. Represents the neighboring cells in the neighborhood of cell u; when δ is much greater than 0 and the neighborhood r of cell u is greater than 0, the neighboring cells in the neighborhood of cell u are greater than 0. u When the set is empty, the soft quantization described above reverts to the original binary definition.

Images