An automatic driving heterogeneous collaborative domain adaptation method based on feature decoupling and reconstruction

Abstract

The application discloses a kind of based on feature decoupling and reconstruction's automatic driving heterogeneous collaborative domain adaptation method, belong to the collaborative perception field of automatic driving.Acquire the features of ego vehicle and collaborative vehicle features obtained by different feature extraction models;Adopt domain-specific mode encoder to extract ego vehicle mode information, and adopt domain-invariant content encoder to cooperate vehicle content information;With ego vehicle mode information and collaborative vehicle content information input based on reconstructor, with collaborative vehicle content information as condition dynamically activate internal expert sub-network to carry out feature reorganization, generate the domain adaptation feature after domain with ego vehicle feature;Finally, ego vehicle feature and domain adaptation feature are fused, and collaborative perception result is output by detection head.The application realizes the one-to-many generalization ability of multiple different collaborative vehicles by using only one unified domain adapter through non-invasive plug-in design, while maintaining the performance of existing homogeneous system, significantly reduces the repeated training cost and vehicle deployment overhead.

Description

Technical Field

[0001] This invention belongs to the field of cooperative perception in autonomous driving, and particularly relates to a method for heterogeneous cooperative domain adaptation in autonomous driving based on feature decoupling and reconstruction. Background Technology

[0002] For autonomous vehicles, environmental perception is crucial. However, due to sensor limitations and frequent field-of-view obstruction, the accuracy of single-vehicle perception is limited. Vehicle-to-vehicle (V2V) cooperative perception can overcome these limitations by exchanging and complementing perception information between vehicles, significantly improving perception performance. A complete cooperative perception system consists of three parts: a feature extraction model, a feature fusion module, and a detection head. The feature extraction model extracts BEV features from the raw input, the feature fusion module fuses features from all vehicles, and the detection head outputs the final detection result. In reality, due to technological barriers and privacy concerns, the perception models of different vehicles vary, leading to a domain gap between the transmitted features. Solving this domain gap is one of the key issues in heterogeneous cooperative perception.

[0003] To address the feature domain gap between heterogeneous vehicles, existing technical solutions mainly fall into two categories. The first category is invasive feature domain adaptation methods. These methods require retraining some module parameters in an existing homogeneous cooperative perception system. For example, they might align the features of the cooperative vehicle to the vehicle domain by fixing the vehicle model and retraining the cooperative vehicle feature extraction model, or retrain different feature fusion modules for different cooperative vehicles. The second category is non-invasive feature domain adaptation methods. These methods do not modify the existing homogeneous cooperative perception system but instead design an independent, plug-and-play domain adaptor to align the features of the cooperative vehicle to the vehicle's feature domain space. However, the domain adaptors designed by these existing methods (such as those by Lu B et al., Luo T et al., Gao X et al., and Xia Y et al.) are essentially one-to-one; that is, for each new cooperative vehicle, a dedicated domain adaptor needs to be retrained and deployed.

[0004] The aforementioned existing technologies have significant shortcomings. Intrusive methods disrupt the original parameters of established homogeneous cooperative perception systems, making them difficult to apply in practice. Non-intrusive methods, due to their one-to-one design paradigm, require retraining and deploying a dedicated domain adaptor for each of the hundreds or thousands of different vehicle perception models in reality. This leads to high training costs and significant deployment pressure on resource-constrained in-vehicle equipment. Designing a plug-and-play, non-intrusive feature domain adaptor capable of many-to-one generalization is a key challenge. An ideal solution should avoid disrupting existing homogeneous perception systems and require only one-time training and deployment to adapt to different cooperative vehicles, thereby fundamentally solving the high costs and deployment pressure caused by repeated training and adapter stacking. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method for adapting heterogeneous cooperative domains in autonomous driving based on feature decoupling and reconstruction. This method decouples the features of cooperative vehicles into domain-invariant content information and domain-specific pattern information, and utilizes a dynamic reconstructor based on a hybrid expert model to reconstruct the content of any cooperative vehicle with the autonomous vehicle's pattern. This enables a multi-to-one generalized domain adaptation that can adapt to multiple unknown heterogeneous vehicles using only a single model.

[0006] The technical solution adopted in this invention is as follows:

[0007] Firstly, this invention proposes a heterogeneous cooperative domain adaptation method for autonomous driving based on feature decoupling and reconstruction, applied to the cooperative perception of the autonomous vehicle and several cooperative vehicles in autonomous driving. The method includes:

[0008] S1. Obtain the features of the autonomous vehicle and one or more cooperative vehicles obtained from different feature extraction models;

[0009] S2. Use a domain-specific pattern encoder to decouple the features of the autonomous vehicle and extract the domain-specific autonomous vehicle pattern information that is related to the feature extraction model; use a domain-invariant content encoder to decouple the features of the cooperative vehicle and extract the scene-related and domain-invariant cooperative vehicle content information.

[0010] S3. Input the autonomous vehicle mode information and the cooperative vehicle content information into a reconstructor constructed based on a hybrid expert model; in the reconstructor, one or more expert sub-networks are dynamically activated with the cooperative vehicle content information as a condition, and the activated expert sub-networks reconstruct the autonomous vehicle mode information and the cooperative vehicle content information respectively. By combining all the reconstruction results, a domain-adapted feature that is located in the same feature domain as the autonomous vehicle feature is obtained.

[0011] S4. Fuse the vehicle features with the domain-adapted features, input the fused features into the detection head, and obtain the collaborative perception results.

[0012] Furthermore, the domain-invariant content encoder, the domain-specific pattern encoder, and the reconstructor are obtained through joint training, and the loss functions used for training include cross-domain content consistency loss and content-pattern orthogonality loss.

[0013] The cross-domain content consistency loss refers to: using point cloud data collected by the autonomous vehicle and point cloud data collected by each cooperative vehicle as input samples; for any sample point cloud, inputting it into both the autonomous vehicle feature extraction model and the cooperative vehicle feature extraction model to obtain a pair of features; traversing all sample point clouds to obtain several feature pairs; and constraining the feature pairs to minimize the differences between the obtained content information after each feature is processed by a domain-invariant content encoder.

[0014] The content-pattern orthogonal loss refers to: inputting the autonomous vehicle features into a domain-invariant content encoder and a domain-specific pattern encoder to obtain autonomous vehicle content information and autonomous vehicle pattern information; inputting the cooperative vehicle features into a domain-invariant content encoder and a domain-specific pattern encoder to obtain cooperative vehicle content information and cooperative vehicle pattern information; calculating the first cosine similarity between the global vector of the autonomous vehicle content information and the global vector of the autonomous vehicle pattern information; calculating the second cosine similarity between the global vector of the cooperative vehicle content information and the global vector of the cooperative vehicle pattern information; and minimizing the sum of the squares of the first cosine similarity and the squares of the second cosine similarity.

[0015] Furthermore, the loss function used for training also includes reconstruction loss and detection task loss.

[0016] Furthermore, the domain-invariant content encoder employs a deformable attention mechanism.

[0017] Furthermore, the domain-specific pattern encoder employs a multi-scale feature pyramid network.

[0018] Furthermore, the reconfigurator includes a gating network and multiple parallel expert subnetworks; the collaborative vehicle content information is input into the gating network, the gating network generates dynamic weights corresponding to each expert subnetwork, and one or more expert subnetworks are activated based on the dynamic weights.

[0019] Furthermore, each expert subnetwork employs a multi-head cross-attention mechanism, using collaborative vehicle content information as the query and autonomous vehicle mode information as the key and value, to perform feature recombination.

[0020] Secondly, the present invention proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for heterogeneous cooperative domain adaptation of autonomous driving based on feature decoupling and reconstruction.

[0021] Thirdly, the present invention proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for heterogeneous cooperative domain adaptation of autonomous driving based on feature decoupling and reconstruction.

[0022] Fourthly, the present invention provides a computer electronic device, including a memory and a processor;

[0023] The memory is used to store computer programs;

[0024] The processor is configured to implement the above-described method for adapting heterogeneous cooperative domains for autonomous driving based on feature decoupling and reconstruction when executing the computer program.

[0025] The beneficial effects of this invention are:

[0026] This invention, through feature decoupling and dynamic reconstruction based on a hybrid expert model, produces a plug-and-play domain adaptor. It eliminates the need to retrain or modify any existing parameters in the homogeneous cooperative perception system, such as feature extraction models, fusion modules, and detection heads, thus maintaining the integrity and performance of the deployed perception system and achieving true non-intrusive design. The domain adaptor trained once can dynamically adapt features from any unknown cooperative vehicle to the vehicle's feature domain, achieving a many-to-one generalization capability for all vehicles with a single model. This fundamentally avoids the enormous repetitive training overhead of existing technologies that require retraining and deploying a dedicated adapter for each new type of cooperative vehicle. Simultaneously, only a single lightweight domain adaptor plugin needs to be deployed on the vehicle-side, greatly alleviating the pressure on limited computing and storage resources and significantly reducing actual deployment and maintenance costs. In summary, this invention achieves efficient and robust perception support for open, dynamically changing heterogeneous vehicle cooperative networks at the lowest possible ongoing cost, without disrupting existing systems. Attached Figure Description

[0027] Figure 1 This is a flowchart of the heterogeneous cooperative domain adaptation method for autonomous driving based on feature decoupling and reconstruction provided by the present invention.

[0028] Figure 2 This is a schematic diagram of the domain-invariant content encoder provided by the present invention;

[0029] Figure 3 This is a schematic diagram of the domain-specific pattern encoder provided by the present invention;

[0030] Figure 4 This is a schematic diagram of the reconfigurator provided by the present invention;

[0031] Figure 5This is a framework diagram of the autonomous driving heterogeneous cooperative domain adaptation system based on feature decoupling and reconstruction provided by the present invention;

[0032] Figure 6 This is a schematic diagram of a computer electronic device provided by the present invention. Detailed Implementation

[0033] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0034] The accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] like Figure 1 As shown, this invention proposes a heterogeneous cooperative domain adaptation method for autonomous driving based on feature decoupling and reconstruction. This method is applied to the cooperative perception of a self-driving vehicle and several cooperative vehicles in autonomous driving, processing heterogeneous features from different vehicle models to achieve effective fusion. Specifically, in actual vehicle-road cooperative scenarios, self-driving vehicles and cooperative vehicles equipped with different brands of perception systems first exchange their respective BEV features (features obtained from the input point cloud through a feature extraction model) and pose information through vehicle-to-everything (V2X) communication. Subsequently, the self-driving vehicle uses the unified domain adaptor built into this method to decouple and reconstruct all received heterogeneous cooperative vehicle features in real time, processing them into domain-adapted features located in the same feature domain as the self-driving vehicle. Finally, these domain-adapted features are fused with the self-driving vehicle features for the final perception decision.

[0037] This invention mainly includes the following steps:

[0038] S1. Obtain the features of the autonomous vehicle and one or more cooperative vehicles obtained from different feature extraction models.

[0039] Generally, both autonomous vehicles and cooperative vehicles are equipped with environmental perception sensors and positioning devices. Autonomous vehicles use their own sensors to collect raw point cloud data (hereinafter referred to as autonomous vehicle point cloud), while cooperative vehicles also use their own sensors to collect raw point cloud data from their own perspective (hereinafter referred to as cooperative vehicle point cloud).

[0040] Each party uses its own pre-trained and fixed feature extraction model to process its own raw data. Specifically, the self-vehicle's point cloud is input into its feature extraction model, which calculates and generates the self-vehicle's BEV (bird's-eye view) features. Each cooperating vehicle inputs its own collected point cloud into its own feature extraction model (which is different from the self-vehicle's model, and the models of each cooperating vehicle may also be different), and generates its own cooperating vehicle BEV features.

[0041] In a vehicle-to-everything (V2X) communication environment, each cooperating vehicle sends its calculated BEV (Battery Electric Vehicle) features and its own pose information obtained through a positioning device to its own vehicle. Simultaneously, the own vehicle receives the BEV features and corresponding pose information from one or more cooperating vehicles via its communication module.

[0042] S2. Feature decoupling: Extracting vehicle mode information and cooperative vehicle content information.

[0043] A domain-specific pattern encoder is used to decouple the features of autonomous vehicles and extract domain-specific autonomous vehicle pattern information that is related to the feature extraction model; a domain-invariant content encoder is used to decouple the features of cooperative vehicles and extract scene-related and domain-invariant cooperative vehicle content information.

[0044] This step separates the style-related content to a specific model from the content related to the scene within the features. The specific process involves two parallel paths depending on the encoding objective:

[0045] Path 1: Extracting invariant content information from the feature extraction domain of the collaborative vehicle

[0046] This task is accomplished by a domain-invariant content encoder, which is built on a deformable attention mechanism and is specifically designed to extract texture variations introduced by a specific model from features and capture the inherent spatial semantics of the scene, such as object occupancy and road structure.

[0047] like Figure 2 As shown, the encoder receives features of the cooperative vehicle. As input, its internal processing comprises two cooperating branches. In the weight calculation branch: features first pass through a linear layer to obtain positional encoding, which is then added to the original features, followed by another linear layer and normalized using Softmax to calculate the attention weights assigned to each feature to be sampled. In the feature sampling branch: features pass through another independent linear layer to predict a set of dynamic 2D spatial offsets. Based on these offsets, more informative contextual features are sampled from the input features. Finally, the sampled features and their corresponding attention weights are input into a cross-attention module for weighted aggregation, ultimately outputting extracted domain-invariant content information independent of the feature extraction model. Here, C, H, and W represent the number of channels, height, and width of the features output by the feature extraction model.

[0048] Path 2: Extracting specific pattern information from the vehicle feature domain

[0049] This task is accomplished by a domain-specific pattern encoder that employs a multi-scale feature pyramid network structure to capture the unique style of the feature extraction model that renders the scene content, namely the unique distribution and texture details it introduces.

[0050] like Figure 3 As shown, the encoder receives vehicle features. As input, the input features are first downsampled twice by a convolutional layer to obtain feature maps with three different scales: large, medium, and small. , , The features at these three scales are then input into separate ConvNext modules for deep feature extraction, fully capturing the representation of pattern information at each scale. Subsequently, convolutional layers upsample the medium and small-scale features to their original resolution, and all three scales' features are concatenated together. Finally, a convolutional layer fuses and integrates these features, outputting domain-specific pattern information that comprehensively represents the characteristics of the vehicle feature extraction model. .

[0051] By decoupling features, two intermediate representations with different properties were obtained: pure scene content information from the features of the cooperating vehicle, and pattern information from the features of the autonomous vehicle, which represents the style of the model.

[0052] S3, Dynamic Reconstruction: Generates features adapted to the domain.

[0053] The autonomous vehicle mode information and the cooperative vehicle content information are input into a reconstructor constructed based on a hybrid expert model. In the reconstructor, one or more expert sub-networks are dynamically activated based on the cooperative vehicle content information. The activated expert sub-networks reconstruct the autonomous vehicle mode information and the cooperative vehicle content information respectively. The results of all reconstructions are combined to obtain the domain-adapted features that are located in the same feature domain as the autonomous vehicle features.

[0054] like Figure 4 As shown, in a specific embodiment of the present invention, the reconfigurator consists of a gating network and multiple parallel expert subnetworks, and its workflow is as follows: The gating network first receives the cooperative vehicle content information. As input conditions. This lightweight convolutional block is based on Calculate and output a set of dynamic importance weights, where each weight corresponds to an expert subnetwork. These weights determine whether each expert subnetwork is activated and to what extent it should be activated when processing specific collaborative vehicle content. All expert subnetworks share the same network structure but have independent parameters.

[0055] Each expert subnetwork simultaneously receives content information from the collaborative vehicle. Information on self-driving mode As input, firstly, the content is projected through a learnable linear projection layer. Mapping to query vectors, converting patterns These are mapped to key and value vectors, respectively. The queries, keys, and values ​​are then input into a multi-head cross-attention module, allowing content information to interact and align within the context of pattern information. This enables conditional reorganization of information and outputs preliminary fused features. The attention output, after layer normalization and residual connections, is then fed into a feedforward network for nonlinear transformation and feature refinement, ultimately yielding the output of the expert sub-network.

[0056] The outputs of all expert subnetworks are weighted and summed based on the dynamic weights generated by the gating network. By dynamically combining different expert capabilities based on content, the reconstructor can flexibly recombine arbitrary cooperative vehicle content with autonomous vehicle patterns, ultimately outputting domain-adapted features consistent with the autonomous vehicle's feature domain distribution. .

[0057] By combining domain-invariant content information from unknown cooperative vehicles with vehicle-specific pattern information, a domain-adapted post-feature that can be seamlessly integrated with vehicle features is generated, thereby achieving the core objective of adapting a unified model to multiple heterogeneous vehicles.

[0058] S4. Fuse the vehicle features with the domain-adapted features, input the fused features into the detection head, and obtain the collaborative perception results.

[0059] The input to this step consists of two parts: first, the original vehicle features generated by the vehicle feature extraction model; and second, the domain-adapted features output from step S3, which have been aligned with the vehicle feature domain. These two sets of features are input together into a feature fusion module. This invention does not limit the feature fusion method; it can employ fusion strategies based on attention mechanisms, convolutional operations, or other deep learning methods to generate a fused feature map. This fused feature map contains environmental perception information aligned between the vehicle's perspective and the perspectives of all cooperating vehicles. The fused feature map is then input to a detection head, which is a network module designed for specific downstream perception tasks, such as 3D object detection and semantic segmentation. Taking object detection as an example, the detection head decodes the fused features, predicts the category, location, size, and orientation of various objects in the environment, and outputs the final collaborative perception result.

[0060] In one specific embodiment of the present invention, the domain-invariant content encoder, the domain-specific pattern encoder, and the reconstructor constitute the domain adaptor of the present invention. The learnable domain adaptor, together with the pre-trained feature extraction model, the feature fusion module, and the detection head, constitute the present invention. The loss function used for training the domain adaptor includes cross-domain content consistency loss, content-pattern orthogonality loss, reconstruction loss, and detection task loss.

[0061] (1) Cross-domain content consistency loss

[0062] The cross-domain content consistency loss refers to: using point cloud data collected by the autonomous vehicle and point cloud data collected by each cooperative vehicle as input samples; for any sample point cloud, simultaneously inputting it into the autonomous vehicle feature extraction model and the cooperative vehicle feature extraction model to obtain a pair of features; traversing all sample point clouds to obtain several feature pairs. The differences between the content information obtained after each feature in a constrained feature pair is minimized by passing it through a domain-invariant content encoder.

[0063] This loss primarily supervises the domain-invariant content encoder, forcing it to extract model-related style or texture from the features generated by the different feature extraction models, and extracting purely scene-invariant content information.

[0064] During the forward propagation of training, for each pair of features corresponding to a batch of data... Encoder that keeps the same content field input Two outputs are obtained. The loss function minimizes the Euclidean distance between the two content outputs.

[0065] Its mathematical expression is:

[0066]

[0067] in, This is a loss of cross-domain content consistency. It is the square of the L2 norm. It is a content encoder.

[0068] (2) Content-pattern orthogonal loss

[0069] To enhance the robustness of feature separation and prevent the information extracted by the content encoder from containing pattern information, or vice versa, ideally, the two should be independent of each other. To achieve this, this invention proposes a content-pattern orthogonal loss, which minimizes the correlation between the global vector of content information extracted by the domain-invariant content encoder and the global vector of pattern information extracted by the domain-specific pattern encoder. By minimizing the mutual information between content and pattern, the two encoders are forced to focus on their respective tasks, achieving more robust and thorough decoupling.

[0070] During the forward propagation of training, for a batch of data, the self-content is obtained through the content encoder and the pattern encoder, respectively. Autonomous mode Collaborative vehicle content Collaborative vehicle mode First, global average pooling is performed on the four elements to obtain their respective global feature vectors. Then, the first cosine similarity between the global vector of the autonomous vehicle's content information and the global vector of the autonomous vehicle's mode information is calculated; the second cosine similarity between the global vector of the cooperative vehicle's content information and the global vector of the cooperative vehicle's mode information is calculated. The loss function is defined as the sum of the squares of the first cosine similarity and the second cosine similarity. Minimizing the loss forces the global vectors of the content information and the mode information to tend to be orthogonal in the feature space, thereby minimizing the information penetration between the two.

[0071] Its mathematical expression is:

[0072]

[0073] in, It is content-pattern orthogonal loss. It is global average pooling, used to obtain a global vector.

[0074] (3) Reconstruction losses

[0075] The reconstruction loss is used to directly supervise the output quality of the reconstructor, ensuring that it can accurately reconstruct the decoupled cooperative vehicle content and the autonomous vehicle pattern into features that conform to the distribution of the autonomous vehicle feature domain.

[0076] During the forward propagation of training, for a batch of data, the reconstructor outputs the adapted features in the domain. The supervisory signal for this loss, i.e., the ground truth, is the feature obtained by inputting the point cloud collected by the cooperative vehicle into the feature extraction model of the autonomous vehicle. The reconstruction loss is... and The mean square error between them.

[0077] Its mathematical expression is:

[0078]

[0079] in, It is a reconstruction loss. It is the mean square error.

[0080] (4) Detection task loss

[0081] Ensure that the optimization of feature extraction, domain adaptation, feature fusion, and detection head ultimately serves to improve the performance of downstream collaborative perception tasks, and avoid the domain adaptation process becoming disconnected from the final goal.

[0082] This loss is the output loss of the detection head, typically consisting of two parts: classification loss and regression loss. The classification loss uses Focal Loss, and the regression loss uses Smooth L1 Loss, used to regress parameters such as the position, size, and orientation of the target bounding box. This loss can be designed according to the specific downstream task, and will not be elaborated further in this invention.

[0083] (5) Total loss

[0084] In joint training, the contributions of the above-mentioned losses are balanced to jointly optimize all learnable parameters. The total loss is a weighted sum of the four losses, and the preset weights of each loss are used to adjust the relative importance of different loss terms during training, which can be designed according to the actual situation. By minimizing the total loss through backpropagation, the model can simultaneously learn accurate domain adaptation and excellent final perception performance.

[0085] After training, the domain adaptor is used as a plug-and-play, non-intrusive unified component in actual autonomous driving cooperative perception scenarios. When the vehicle needs to perform cooperative perception with surrounding vehicles while driving, its workflow is as follows:

[0086] Step 1: The autonomous vehicle and each cooperating vehicle use their respective LiDAR and other sensors to collect point cloud data and generate their own BEV features through their fixed feature extraction models. The cooperating vehicles then send their BEV features and GPS pose information to the autonomous vehicle via vehicle-to-everything (V2X) communication.

[0087] Step 2: The autonomous vehicle calculates the coordinate transformation matrix based on the received poses of the cooperating vehicle and the autonomous vehicle, and uniformly transforms the features of the cooperating vehicle to the autonomous vehicle coordinate system.

[0088] Step 3: Directly invoke the deployed domain adaptor to process features from any cooperative vehicle:

[0089] The pattern encoder in the domain adaptor parses the vehicle features, extracting domain-specific vehicle pattern information relevant to the feature extraction model. For each feature sent by a cooperative vehicle, the domain adaptor uses its content encoder to extract scene-related, domain-invariant cooperative vehicle content information. The reconstructor, conditioned on the cooperative vehicle content information, dynamically activates its internal expert subnetwork, recombining the vehicle pattern information with the cooperative vehicle content information to output domain-adapted features adapted to the vehicle feature domain. This process is equally effective for any newly emerging cooperative vehicle models not seen during training.

[0090] Step 4: The autonomous vehicle inputs its own original features and all the domain-adapted features of the cooperating vehicles into the feature fusion module to generate a fused feature map. Finally, this feature map is input into the detection head, which outputs the final environmental perception result to support autonomous driving decisions.

[0091] In this embodiment, the proposed method is verified. Regarding the hardware environment, the publicly available V2V4Real and V2X-Real datasets are used for training and validation, and the average precision (AP) is employed. As an evaluation metric, it is recommended that the system computing platform be configured with an NVIDIA RTX 3090 or higher performance GPU with at least 24GB of video memory, and implemented using the PyTorch deep learning framework.

[0092] In object detection tasks, Average Precision (AP) is a core performance metric. It's first determined by the Intersection over Union (IoU) threshold to classify detected samples as positive or negative. Specifically, if the IoU between the predicted bounding box and the ground truth bounding box is greater than a preset threshold, the detection is considered a correct detection (true positive); otherwise, it's considered a false positive. By setting different classification confidence thresholds, a precision-recall curve can be plotted. Precision measures the proportion of samples predicted as positive by the model that are actually positive, while recall measures the proportion of all true positives correctly detected by the model. Average precision is defined as the area under this precision-recall curve.

[0093] The comparison methods include PLDA, PnPDA, GenComm, STAMP, MPDA, and PolyInter. Among them, PLDA proposes a domain adaptor based on MLN to achieve feature domain adaptation while protecting privacy; PnPDA proposes a plug-and-play domain adaptor to align heterogeneous features; GenComm proposes a generative communication mechanism to achieve feature reconstruction; STAMP designs paired adaptors and restorers to transform features into a common space; MPDA proposes to achieve feature domain adaptation using generative adversarial networks; and PolyInter proposes a polymorphic interpreter to improve the perception accuracy of immutable heterogeneous collaboration.

[0094] The test results are shown in Tables 1, 2, and 3.

[0095] Table 1 Comparison of one-to-one adaptability

[0096]

[0097] Table 2 Comparison of many-to-one adaptability in known domains

[0098]

[0099] Table 3 Comparison of zero-sample generalization ability in the unknown domain

[0100]

[0101] As can be seen, the method proposed in this invention is significantly superior to existing methods in one-to-one domain adaptation, many-to-one domain adaptation in known domains, and zero-sample generalization ability in unknown domains.

[0102] It should also be noted that the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides an autonomous driving heterogeneous cooperative domain adaptation system based on feature decoupling and reconstruction, corresponding to the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction provided in the above embodiments, such as... Figure 5 As shown, it includes:

[0103] The feature acquisition module is used to acquire the features of the autonomous vehicle obtained from different feature extraction models, as well as one or more features of the cooperating vehicle.

[0104] A feature decoupling module is connected to the feature acquisition module. The feature decoupling module includes a domain-specific pattern encoder and a domain-invariant content encoder; wherein:

[0105] A domain-specific pattern encoder is used to decouple the vehicle features and extract domain-specific vehicle pattern information related to the feature extraction model.

[0106] A domain-invariant content encoder is used to decouple the features of the collaborative vehicle and extract scene-related, domain-invariant collaborative vehicle content information.

[0107] The reconstruction module is connected to the feature decoupling module. The reconstruction module is a reconstructor built based on a hybrid expert model. It is used to receive the autonomous vehicle mode information and the cooperative vehicle content information, and dynamically activate one or more expert sub-networks within it based on the cooperative vehicle content information. The activated expert sub-networks reconstruct the autonomous vehicle mode information and the cooperative vehicle content information respectively. The reconstruction results are combined to output the domain-adapted features that are located in the same feature domain as the autonomous vehicle features.

[0108] The fusion and perception module, connected to the feature acquisition module and the reconstruction module, is used to fuse the vehicle features with the domain-adapted features, and input the fused features into the detection head to obtain the collaborative perception result.

[0109] It is understood that the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction in the above embodiments can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction provided in the above embodiments. This product includes a computer program / instruction, which, when executed by a processor, can implement the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction as described in the above embodiments.

[0110] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the feature decoupling and reconstruction-based heterogeneous cooperative domain adaptation method for autonomous driving provided in the above embodiments, such as... Figure 6 As shown, it includes a memory and a processor;

[0111] The memory is used to store computer programs;

[0112] The processor is configured to implement the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction in the above embodiments when executing the computer program.

[0113] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0114] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction in the above embodiments.

[0115] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0116] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0117] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0118] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for heterogeneous cooperative domain adaptation in autonomous driving based on feature decoupling and reconstruction, applied to the cooperative perception of a self-driving vehicle and several cooperative vehicles in autonomous driving, characterized in that the method... include: S1. Obtain the features of the autonomous vehicle and one or more cooperative vehicles obtained from different feature extraction models; S2. Use a domain-specific pattern encoder to decouple the features of the autonomous vehicle and extract the domain-specific autonomous vehicle pattern information that is related to the feature extraction model; use a domain-invariant content encoder to decouple the features of the cooperative vehicle and extract the scene-related and domain-invariant cooperative vehicle content information. The domain-invariant content encoder employs a deformable attention mechanism. The domain-specific pattern encoder employs a multi-scale feature pyramid network. S3. Input the autonomous vehicle mode information and the cooperative vehicle content information into a reconstructor constructed based on a hybrid expert model; in the reconstructor, one or more expert sub-networks are dynamically activated with the cooperative vehicle content information as a condition, and the activated expert sub-networks reconstruct the autonomous vehicle mode information and the cooperative vehicle content information respectively. By combining all the reconstruction results, a domain-adapted feature that is located in the same feature domain as the autonomous vehicle feature is obtained. The reconstructor includes a gating network and multiple parallel expert subnetworks. The collaborative vehicle content information is input into the gating network, which generates dynamic weights corresponding to each expert subnetwork. One or more expert subnetworks are activated based on the dynamic weights. Each expert subnetwork uses a multi-head cross-attention mechanism, using the collaborative vehicle content information as the query and the autonomous vehicle mode information as the key and value, to perform feature reconstructing. S4. Fuse the vehicle features with the domain-adapted features, input the fused features into the detection head, and obtain the collaborative perception results.

2. The method for heterogeneous cooperative domain adaptation in autonomous driving based on feature decoupling and reconstruction according to claim 1, characterized in that, The domain-invariant content encoder, the domain-specific pattern encoder, and the reconstructor are obtained through joint training. The loss functions used for training include cross-domain content consistency loss and content-pattern orthogonality loss. The cross-domain content consistency loss refers to: using point cloud data collected by the autonomous vehicle and point cloud data collected by each cooperative vehicle as input samples; for any sample point cloud, inputting it into both the autonomous vehicle feature extraction model and the cooperative vehicle feature extraction model simultaneously to obtain a pair of features; By traversing all sample point clouds, several feature pairs are obtained; after each feature in a constrained feature pair is processed by a domain-invariant content encoder, the difference between the resulting content information is minimized. The content-pattern orthogonal loss refers to: inputting the autonomous vehicle features into a domain-invariant content encoder and a domain-specific pattern encoder to obtain autonomous vehicle content information and autonomous vehicle pattern information; inputting the cooperative vehicle features into a domain-invariant content encoder and a domain-specific pattern encoder to obtain cooperative vehicle content information and cooperative vehicle pattern information; calculating the first cosine similarity between the global vector of the autonomous vehicle content information and the global vector of the autonomous vehicle pattern information; calculating the second cosine similarity between the global vector of the cooperative vehicle content information and the global vector of the cooperative vehicle pattern information; and minimizing the sum of the squares of the first cosine similarity and the squares of the second cosine similarity.

3. The method for heterogeneous cooperative domain adaptation in autonomous driving based on feature decoupling and reconstruction according to claim 2, characterized in that, The loss function used for training also includes reconstruction loss and detection task loss.

4. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can implement the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction as described in any one of claims 1 to 3.

6. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the autonomous driving heterogeneous cooperative domain adaptation method based on feature decoupling and reconstruction as described in any one of claims 1 to 3 when executing the computer program.

Images