A model training method, a cross-domain target detection method and an electronic device

Abstract

The application provides a model training method, a cross-domain target detection method and an electronic device, and is used for improving the problem of detection accuracy reduction of a model. The method comprises the following steps: acquiring a labeled source domain data set and an unlabeled target domain data set, the target domain data set comprising a target domain image; calculating a source domain loss of a source domain detection head network in a student model according to the source domain data set; performing target detection on the target domain image by using a teacher model to obtain a target domain pseudo label, the weight parameter of the teacher model being obtained by being valued by the weight parameter of the student model; calculating a target domain loss of a target domain detection head network in the student model according to the target domain pseudo label and the target domain image; calculating an alignment loss of a multi-scale mask alignment module in the student model according to the source domain data set and the target domain data set; and training the student model according to the source domain loss, the target domain loss and the alignment loss to obtain a trained student model.

Description

Technical Field

[0001] This application relates to the technical fields of deep learning, computer vision, and object detection, and more specifically, to a model training method, a cross-domain object detection method, and an electronic device. Background Technology

[0002] In current cross-domain object detection processes, most models use weight-shared detection head networks to classify source and target domain data samples. Although weight-shared detection heads aim to promote the transfer and sharing of cross-domain features, in practical applications, it has been found that because the source domain data has real labeled information, the model often relies too much on the supervised training process of the source domain data samples during joint optimization. This makes the model more biased towards optimizing the source domain data samples and neglects the optimization of the target domain data samples, resulting in a decrease in the quality of pseudo-labels generated by the teacher model and ultimately a decrease in the detection accuracy of the model. Summary of the Invention

[0003] The purpose of this application is to provide a model training method, a cross-domain target detection method, and an electronic device to improve the problem of declining detection accuracy of the model.

[0004] This application provides a model training method, including: acquiring a labeled source domain dataset and an unlabeled target domain dataset, wherein the target domain dataset includes target domain images; calculating the source domain loss of the source domain detection head network in the student model based on the source domain dataset; performing target detection on the target domain images using the teacher model to obtain target domain pseudo-labels, wherein the weight parameters of the teacher model are obtained by assigning values ​​to the weight parameters of the student model; calculating the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-labels and the target domain images; calculating the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset; and training the student model based on the source domain loss, the target domain loss, and the alignment loss to obtain the trained student model. In the implementation of the above scheme, by simultaneously setting up a source domain detection head network for source domain optimization and a target domain detection head network for target domain optimization during the training process of the student model, the source domain optimization and target domain optimization are decoupled during the decoding process. This allows the student model to fully explore the domain feature information with domain invariance in the target domain data samples, enabling the teacher model to more effectively utilize this domain feature information to improve the quality of pseudo-labels, thereby improving the detection accuracy of the student model.

[0005] Optionally, in this embodiment, the source domain dataset includes: source domain images and source domain labels; calculating the source domain loss of the source domain detection head network in the student model based on the source domain dataset includes: extracting features from the source domain images through the backbone network in the student model to obtain a multi-layer feature map of the source domain; performing multi-scale feature fusion on the multi-layer feature map of the source domain through the feature fusion layer in the student model to obtain a multi-scale feature map of the source domain; performing object detection on the multi-scale feature map of the source domain through the source domain detection head network in the student model to obtain a source domain detection result; and determining the loss value between the source domain detection result and the source domain labels as the source domain loss. In the implementation of the above scheme, the multi-scale feature map of the source domain is processed by the source domain detection head network to ensure that the model can extract rich and multi-layer feature information from the source domain data, thereby enabling the student model to optimize the performance and adaptability of the source domain, improving the basic detection capability of the student model in the source domain, so that when this trained model is applied to the target domain, the model can better extract and identify common features, thereby potentially reducing the impact of domain differences and improving the performance of the student model in the target domain.

[0006] Optionally, in this embodiment, the target domain loss of the target domain detection head network in the student model is calculated based on the target domain pseudo-label and the target domain image. This includes: extracting features from the target domain image through the backbone network in the student model to obtain a multi-layer feature map of the target domain; performing multi-scale feature fusion on the multi-layer feature map of the target domain through the feature fusion layer in the student model to obtain a multi-scale feature map of the target domain; performing target detection on the multi-scale feature map of the target domain through the target domain detection head network in the student model to obtain a target domain detection result; and determining the loss value between the target domain detection result and the target domain pseudo-label as the target domain loss. In the implementation of the above scheme, by performing target detection on the multi-scale feature map of the target domain through the target domain detection head network in the student model and determining the loss value between the target domain detection result and the target domain pseudo-label as the target domain loss, the gap between the source domain and the target domain is effectively bridged, enhancing the performance and generalization ability of the student model in the target domain.

[0007] Optionally, in this embodiment, the alignment loss of the multi-scale mask adversarial alignment module in the student model is calculated based on the source domain dataset and the target domain dataset, including: obtaining a random mask and channel variables, wherein the size of the random mask is the same as the size of the target domain multi-scale feature map, and the number of channels of the channel variables is the same as the number of channels of the target domain multi-scale feature map; calculating the source domain reconstruction features by using the multi-scale mask adversarial alignment module on the random mask, channel variables, and source domain multi-scale feature map; calculating the target domain reconstruction features by using the multi-scale mask adversarial alignment module on the random mask, channel variables, and target domain multi-scale feature map; and calculating the alignment loss by using the source domain reconstruction features and the target domain reconstruction features. In the implementation of the above scheme, since the target domain lacks labeled information and the interference from the background region of the target domain easily leads to misjudgment by the model, the foreground region of the target domain image is easily classified as a negative sample. The backbone network and feature fusion layer in the student model act as feature extractors. During domain adaptive training, these feature extractors often focus more on the features of the source domain data, thus classifying the foreground region of the target domain image as background features and ignoring the foreground information of the target domain image. Therefore, by reconstructing features from the source domain and the target domain, the student model can pay more attention to the features of the foreground region of the target domain image, thereby improving the detection accuracy of the student model for the target domain image.

[0008] Optionally, in this embodiment, the multi-scale mask adversarial alignment module includes: a fully convolutional decoder; the multi-scale feature map of the target domain is calculated by the multi-scale mask adversarial alignment module, including: mask weighting of the random mask, channel variables, and target domain multi-scale feature map to obtain mask-weighted multi-scale features; and mask decoding of the mask-weighted multi-scale features using the fully convolutional decoder to obtain target domain reconstructed features. In the implementation of the above scheme, the random mask weighting and decoding in the multi-scale mask adversarial alignment module reconstruct multi-scale features, combined with adversarial training methods, help the student model escape local suboptimal results caused by erroneous pseudo-labels, thereby achieving pixel-level feature alignment and ultimately improving the detection accuracy of the student model for the target domain image.

[0009] Optionally, in this embodiment, the multi-scale mask adversarial alignment module further includes: a gradient flip layer and a domain discriminator; calculating the source domain reconstruction features and the target domain reconstruction features includes: using the gradient flip layer to align the source domain reconstruction features and the target domain reconstruction features to obtain aligned source domain reconstruction features and aligned target domain reconstruction features; and using the domain discriminator to calculate the alignment loss between the aligned source domain reconstruction features and the aligned target domain reconstruction features. In the implementation of the above scheme, the gradient flip layer and domain discriminator in the adversarial training method help the student model escape local suboptimal results caused by erroneous pseudo-labels, thereby achieving pixel-level feature alignment and ultimately improving the detection accuracy of the student model for the target domain image.

[0010] This application also provides a cross-domain object detection method, including: acquiring an image to be processed; and performing object detection on the image to be processed using the student model described above. In the implementation of the above scheme, by simultaneously setting up a source domain detection head network for source domain optimization and a target domain detection head network for target domain optimization, object detection is performed on the image to be processed. This decouples source domain optimization and target domain optimization during the decoding process, allowing the student model to fully exploit the domain feature information with domain invariance in the target domain data samples, effectively improving the accuracy of object detection on the image to be processed.

[0011] This application also provides a model training apparatus, comprising: a domain dataset acquisition module for acquiring labeled source domain datasets and unlabeled target domain datasets, wherein the target domain dataset includes target domain images; a source domain loss calculation module for calculating the source domain loss of the source domain detection head network in the student model based on the source domain dataset; a domain pseudo-label acquisition module for performing target detection on the target domain images using a teacher model to obtain target domain pseudo-labels, wherein the weight parameters of the teacher model are obtained by assigning values ​​to the weight parameters of the student model; a target loss calculation module for calculating the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-labels and the target domain images; an alignment loss calculation module for calculating the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset; and a student model training module for training the student model based on the source domain loss, the target domain loss, and the alignment loss to obtain the trained student model.

[0012] Optionally, in this embodiment, the source domain dataset includes: source domain images and source domain labels; the source domain loss calculation module includes: a source domain feature extraction submodule, used to extract features from the source domain images through the backbone network in the student model to obtain a source domain multi-layer feature map; a source domain feature fusion submodule, used to perform multi-scale feature fusion on the source domain multi-layer feature map through the feature fusion layer in the student model to obtain a source domain multi-scale feature map; a source domain object detection submodule, used to perform object detection on the source domain multi-scale feature map through the source domain detection head network in the student model to obtain a source domain detection result; and a source domain loss determination submodule, used to determine the loss value between the source domain detection result and the source domain label as the source domain loss.

[0013] Optionally, in this embodiment, the target loss calculation module includes: a target domain feature extraction submodule, used to extract features from the target domain image through the backbone network in the student model to obtain a multi-layer feature map of the target domain; a target domain feature fusion submodule, used to perform multi-scale feature fusion on the multi-layer feature map of the target domain through the feature fusion layer in the student model to obtain a multi-scale feature map of the target domain; a target domain target detection submodule, used to perform target detection on the multi-scale feature map of the target domain through the target domain detection head network in the student model to obtain a target domain detection result; and a target domain loss determination submodule, used to determine the loss value between the target domain detection result and the target domain pseudo-label as the target domain loss.

[0014] Optionally, in this embodiment, the alignment loss calculation module includes: a mask variable acquisition submodule, used to acquire a random mask and channel variables, wherein the size of the random mask is the same as the size of the target domain multi-scale feature map, and the number of channels of the channel variables is the same as the number of channels of the target domain multi-scale feature map; a reconstruction feature acquisition submodule, used to calculate the random mask, channel variables, and source domain multi-scale feature map through the multi-scale mask adversarial alignment module to obtain source domain reconstruction features; a scale feature acquisition submodule, used to calculate the random mask, channel variables, and target domain multi-scale feature map through the multi-scale mask adversarial alignment module to obtain target domain reconstruction features; and an alignment loss acquisition submodule, used to calculate the source domain reconstruction features and target domain reconstruction features to obtain the alignment loss.

[0015] Optionally, in this embodiment, the multi-scale mask adversarial alignment module includes: a fully convolutional decoder; and a scale feature acquisition submodule, including: a feature mask weighting unit, used to perform mask weighting on a random mask, channel variables, and multi-scale feature maps of the target domain to obtain mask-weighted multi-scale features; and a feature mask decoding unit, used to perform mask decoding on the mask-weighted multi-scale features using the fully convolutional decoder to obtain reconstructed features of the target domain.

[0016] Optionally, in this embodiment, the multi-scale mask adversarial alignment module further includes: a gradient flip layer and a domain discriminator; the alignment loss acquisition submodule includes: a domain feature alignment unit, used to use the gradient flip layer to perform feature alignment on the source domain reconstruction features and the target domain reconstruction features to obtain aligned source domain reconstruction features and aligned target domain reconstruction features; and an alignment loss calculation unit, used to use the domain discriminator to calculate the alignment loss between the aligned source domain reconstruction features and the aligned target domain reconstruction features.

[0017] This application also provides a cross-domain target detection device, including: a processing image acquisition module for acquiring an image to be processed; and an image target detection module for performing target detection on the image to be processed using the student model described above.

[0018] This application also provides an electronic device, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform the methods described above.

[0019] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the methods described above.

[0020] This application also provides a computer program product, including: a computer program or computer instructions, which are executed by a processor to perform the method described above. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the embodiments of this application, and therefore should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The diagram shown is a flowchart of the model training method provided in an embodiment of this application;

[0023] Figure 2 The illustration shows a schematic diagram of the dataset processing during the training process provided in an embodiment of this application;

[0024] Figure 3 The diagram shown is a flowchart of a cross-domain target detection method provided in an embodiment of this application.

[0025] Figure 4 The diagram shown is a schematic flowchart of the model training apparatus provided in an embodiment of this application;

[0026] Figure 5 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in the embodiments of this application are for illustrative and descriptive purposes only and are not intended to limit the protection scope of the embodiments of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the embodiments of this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of the embodiments of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0028] Furthermore, the described embodiments are merely a part of the embodiments of this application, and not all of them. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely to illustrate selected embodiments of this application.

[0029] It is understood that the terms "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different. In the description of the embodiments of this application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. The term "multiple" refers to two or more (including two), and similarly, "multiple groups" refers to two or more groups (including two groups).

[0030] It should be noted that the model training method and cross-domain object detection method provided in this application can be executed by electronic devices. Here, electronic devices refer to device terminals or servers with the function of executing computer programs. Device terminals include, for example, smartphones, personal computers, tablets, personal digital assistants, or mobile internet devices. Servers refer to devices that provide computing services through a network. Servers include, for example, x86 servers and non-x86 servers. Non-x86 servers include, for example, mainframes, minicomputers, and UNIX servers.

[0031] In some cross-domain object detection scenarios, pseudo-label deep-domain adaptive cross-domain object detection techniques can be used to effectively utilize unlabeled target domain data. This primarily involves adding a teacher model to the object detector to guide target domain training. Specifically, the teacher model adds pseudo-labels to the unlabeled target domain data in the student model. These pseudo-labels guide the training of the student model, which uses a weight-shared detection head network, in the target domain. This pseudo-label deep-domain adaptive cross-domain object detection technique improves model performance by utilizing unlabeled target domain data. Its core idea is to use a model trained in the source domain to predict target domain data, and then use these predictions as pseudo-labels for further training or fine-tuning. However, in these scenarios, it has been found that because the source domain data contains real-world annotations, the teacher and student models often rely too heavily on the supervised training process of the source domain data samples during joint optimization. This causes the model to favor optimizing the source domain data samples while neglecting the target domain data samples, resulting in a decrease in the quality of the pseudo-labels generated by the teacher model and ultimately a decrease in the model's detection accuracy.

[0032] Please see Figure 1 The illustrated flowchart shows the model training method provided in this application embodiment. To address the aforementioned problems, this application embodiment simultaneously sets up a source domain detection head network for source domain optimization and a target domain detection head network for target domain optimization during the training process of the student model. This decouples source domain optimization and target domain optimization during the decoding process, allowing the student model to fully exploit the domain feature information with domain invariance in the target domain data samples. This application embodiment provides a model training method to describe the training process of the above student model, which may include:

[0033] Step S110: Obtain the labeled source domain dataset and the unlabeled target domain dataset. The target domain dataset includes: target domain images.

[0034] For example, in implementing step S110 above, after obtaining the labeled source domain dataset, source domain images and source domain labels can be parsed from the source domain dataset, and the source domain images can be preprocessed to obtain preprocessed source domain images. Then, strong data augmentation can be performed on the preprocessed source domain images to obtain strongly augmented source domain images. The preprocessing here includes, but is not limited to, operations such as data normalization and scaling. The strong augmentation operations include, but are not limited to, at least two or more of the following operations: rotation, random scaling, horizontal flipping, cropping, Gaussian noise, and random contrast enhancement. After obtaining the unlabeled target domain dataset, target domain images can be parsed from the target domain dataset, and then weak data augmentation can be performed on the target domain images to obtain weakly augmented target domain images. The weak augmentation here can be one of rotation, random scaling, horizontal flipping, cropping, Gaussian noise, and random contrast enhancement. The weakly augmented target domain images are used by the teacher model to generate target domain pseudo-labels.

[0035] Please see Figure 2 The diagram illustrates the processing of the dataset provided in this application's embodiment during the training process. The dataset includes the labeled source domain dataset and the unlabeled target domain dataset, such as source domain images in the source domain dataset and target domain images in the target domain dataset. Both strong and weak augmentation refer to data augmentation of samples. The difference lies in the degree to which they increase the diversity of the input data. Strong augmentation uses multiple methods to enrich the diversity of the input data, thus improving the robustness of the model. Weak data augmentation uses a single method, making it easier for the teacher model to learn and identify positive samples. In the above model training method, strong augmentation is used to increase the robustness of the student model, making it more robust, while weak augmentation enables the teacher model to more accurately identify positive sample targets, thereby improving the accuracy of the target domain pseudo-label.

[0036] Step S120: Calculate the source domain loss of the source domain detection head network in the student model based on the source domain dataset.

[0037] Understandably, since the process of training the student model using the source domain dataset and calculating the source domain loss is quite complex, the process of calculating the source domain loss will be described in detail below for clarity.

[0038] Step S130: Target detection is performed on the target domain image using the teacher model to obtain target domain pseudo-labels. The weight parameters of the teacher model are obtained by assigning values ​​to the weight parameters of the student model.

[0039] Both the teacher model and the student model can be single-stage detection networks. Specifically, they can be neural network models built with a Fully Convolutional One-Stage Object Detection (FCOS) single-stage detection network as the baseline detector, including but not limited to: Feature Fusion Single Shot multi-box Detector (FSSD) and YOLO models.

[0040] Step S140: Calculate the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-label and the target domain image.

[0041] It is understandable that source domain detection head network and target domain detection head network are set up simultaneously during the training of the student model, so that source domain optimization and target domain optimization can be decoupled during the decoding process. This allows the student model to fully explore the specific features in the target domain data samples. These specific features are domain features with domain invariance, specifically foreground region features in the source and target domains, i.e., positive samples that should be classified as target domain.

[0042] Step S150: Calculate the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset.

[0043] Step S160: Train the student model based on the source domain loss, target domain loss, and alignment loss to obtain the trained student model, which is then used for object detection in the image.

[0044] In the implementation of the above scheme, by simultaneously setting up a source domain detection head network for source domain optimization and a target domain detection head network for target domain optimization during the training process of the student model, the source domain optimization and target domain optimization are decoupled during the decoding process. This allows the student model to fully explore the domain feature information with domain invariance in the target domain data samples, enabling the teacher model to more effectively utilize this domain feature information to improve the quality of pseudo-labels, thereby improving the detection accuracy of the student model.

[0045] As an optional implementation of step S120 above, the source domain dataset may include: source domain images and source domain labels; the implementation of calculating the source domain loss of the source domain detection head network in the student model based on the source domain dataset may include:

[0046] Step S121: Extract features from the source domain image using the backbone network in the student model to obtain a multi-layer feature map of the source domain.

[0047] An example implementation of step S121 above is as follows: After obtaining the source domain dataset, source domain images and source domain labels can be parsed from the source domain dataset. Features of the source domain images are then extracted using the backbone network in the student model to obtain multi-layer feature maps of the source domain. These multi-layer feature maps can be three-layer feature maps. The backbone network in the student model can be a residual network (ResNet), such as ResNet22, ResNet38, ResNet50, ResNet101, and ResNet152, etc.

[0048] Step S122: Perform multi-scale feature fusion on the source domain multi-layer feature map through the feature fusion layer in the student model to obtain the source domain multi-scale feature map.

[0049] For example, after obtaining the source domain multi-layer feature map, multi-scale feature fusion can be performed on the source domain multi-layer feature map through the feature fusion layer in the student model to obtain the source domain multi-scale feature map. Here, the source domain multi-scale feature map can be represented as... Specifically, the feature fusion layer in the student model mentioned above can be a Feature Pyramid Network (FPN).

[0050] Step S123: Perform target detection on the multi-scale feature map of the source domain using the source domain detection head network in the student model to obtain the source domain detection results.

[0051] An example of the implementation of step S123 above is as follows: The source domain detection head network may include a classification subnetwork and a regression subnetwork. During the process of object detection on the source domain multi-scale feature map using the source domain detection head network in the student model, the classification subnetwork can be used to predict the category of each pixel in the source domain multi-scale features (the source domain category prediction result can be expressed as...). Alternatively, a regression subnetwork can be used to perform regression prediction based on multi-scale features of the source domain (the source domain regression prediction result can be expressed as...). ) and center perception prediction (the result of center perception prediction can be expressed as Finally, the category prediction results, regression prediction results, and center perception prediction results are determined as the source domain detection results mentioned above.

[0052] Step S124: Determine the loss value between the source domain detection result and the source domain label as the source domain loss.

[0053] Source domain labels refer to the image labels corresponding to the source domain images. Here, source domain labels include, but are not limited to: source domain category labels, source domain regression labels, and source domain center perception labels.

[0054] An example implementation of step S124 above is as follows: The source domain category loss is calculated based on the source domain category label and the source domain category prediction result (which can be expressed as...). ), and calculate the source domain regression loss (which can be expressed as ) based on the source domain regression labels and source domain regression prediction results. Then, based on the source domain center perception label and the source domain center perception prediction result, the source domain center perception loss is calculated (which can be expressed as...). Finally, according to the formula The source domain category loss, source domain regression loss, and source domain center perception loss are calculated to obtain the source domain loss after supervised training. Where x S Represents the source domain image, y s Indicates the source domain tag. This represents the source domain loss between the source domain detection result and the source domain label. Indicates source domain category loss. Indicates the source domain regression loss. This represents the loss of perception at the source domain center.

[0055] As an optional implementation of step S130 above, the above implementation of object detection of the target domain image using the teacher model may include:

[0056] Step S131: Perform weak enhancement on the target domain image to obtain the weakly enhanced target domain image.

[0057] For example, the above step S131 can be implemented by using one of the following methods to weakly enhance the target domain image: rotation, random scaling, horizontal flipping, cropping, Gaussian noise, and random contrast enhancement, to obtain a weakly enhanced target domain image, so that the teacher model can more accurately identify the positive sample target and improve the accuracy of the target domain pseudo label.

[0058] Step S132: Perform target detection on the weakly enhanced target domain image using the teacher model to obtain target domain pseudo-labels.

[0059] For example, the implementation of step S132 described above is as follows: After obtaining the weakly enhanced target domain image, feature extraction can be performed on the weakly enhanced target domain image through the backbone network in the teacher model to obtain a multi-layer feature map of the teacher. Then, multi-scale feature fusion is performed on the multi-layer feature map of the teacher through the feature fusion layer in the teacher model to obtain a multi-scale feature map of the teacher. Finally, target detection is performed on the multi-scale feature map of the teacher through the source domain detection head network in the teacher model to obtain target domain pseudo-labels. The aforementioned target domain pseudo-labels are image labels output by the teacher model. These target domain pseudo-labels include, but are not limited to, category pseudo-labels (which can be represented as...). ), regression pseudo-labels (which can be represented as ) and center-awareness pseudo-labels (which can be represented as ).

[0060] Optionally, the target domain pseudo-labels can be filtered based on the category pseudo-labels, regression pseudo-labels, and center perception pseudo-labels in the target domain pseudo-labels. The filtered target domain pseudo-labels are then used to guide the training of the student model. Specifically, the loss value between the target domain detection result and the filtered target domain pseudo-labels is defined as the target domain loss. This can be achieved using the formula... The Hadamard product is calculated by combining the class pseudo-labels and center perception pseudo-labels output by the teacher model, thereby filtering out low-quality false positive pixels far from the center point. Then, the top σ pixels are sorted and taken as positive samples, while the remaining pixels are taken as negative samples. Where y t This represents the filtered target domain pseudo-label, and topk indicates that the preceding pixels are taken as positive samples. This represents the pseudo-label of the category output by the teaching model. The pseudo-label represents the center perception output of the teaching model, σ is the confidence factor controlling the number of positive and negative samples, and ⊙ represents the Hadamard product. The aforementioned false positive pixels refer to the set of pixels that are actually background regions but are classified as foreground regions. These false positive pixels often have a small portion of the target box (which has the characteristic of being rectangular) that is background, and these regions are relatively far from the center of the box, but have high classification prediction values. The value of these pixels can be reduced by multiplying the classification prediction and the center perception prediction.

[0061] As an optional implementation of step S140 above, the implementation of calculating the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-label and the target domain image may include:

[0062] Step S141: Extract features from the target domain image using the backbone network in the student model to obtain a multi-layer feature map of the target domain.

[0063] An example implementation of step S141 above is as follows: After obtaining the labeled target domain dataset, the target domain image and target domain label can be parsed from the target domain dataset, and the target domain image can be preprocessed to obtain a preprocessed target domain image. Then, the preprocessed target domain image is strongly enhanced to obtain a strongly enhanced target domain image. The preprocessing here includes, but is not limited to, operations such as data normalization and scale adjustment. The strong enhancement operations include, but are not limited to, at least two of the following: rotation, random scaling, horizontal flipping, cropping, Gaussian noise, and random contrast enhancement. Then, feature extraction is performed on the strongly enhanced target domain image through the backbone network in the student model to obtain a multi-layer feature map of the target domain. The multi-layer feature map of the source domain can be a three-layer feature map.

[0064] Step S142: Perform multi-scale feature fusion on the multi-layer feature map of the target domain through the feature fusion layer in the student model to obtain the multi-scale feature map of the target domain.

[0065] Step S143: Perform target detection on the multi-scale feature map of the target domain using the target domain detection head network in the student model to obtain the target domain detection result.

[0066] The implementation principles and methods of steps S142 to S143 are similar to those of steps S122 to S123. Therefore, their implementation principles and methods will not be described here. Please refer to the description of steps S122 to S123.

[0067] Step S144: Determine the loss value between the target domain detection result and the target domain pseudo-label as the target domain loss.

[0068] The target domain detection result is the target detection result output by the student model after the target domain image is input. This target domain detection result includes: the target domain category prediction result (which can be represented as...). ), Target domain regression prediction results (which can be represented as ) and the target domain center perception prediction results (which can be expressed as ).

[0069] For example, the above-mentioned implementation of step S144 is as follows: The target domain loss is the loss of the student model in the unsupervised training process. The target domain loss may include: target domain category loss, target domain regression loss and target domain center perception loss. The calculation process of each loss is described below.

[0070] First, since the pseudo-labels output by the teacher model are not standard one-hot discrete labels, the student model can use a function based on Quality Focal Loss (QFL) as its loss function to estimate the continuous 0-1 quality of the corresponding category. Specifically, according to the formula... The target domain category loss is obtained by calculating the target domain category prediction results based on the category pseudo-labels and the target domain category prediction results. Among them, Indicates the target domain category loss. This represents the target domain category prediction result, y t This represents a target domain pseudo-label, which includes a category pseudo-label.

[0071] Secondly, the Generalized Intersection over Union (GIoU) ​​loss can be used as the loss function for regression loss. Specifically, according to the formula... The regression loss for the target domain is obtained by calculating the regression pseudo-label and the target domain regression prediction results. Among these, y represents the regression loss in the target domain. t This represents the target domain pseudo-label, which includes regression pseudo-labels. GIoU represents the regression prediction result for the target domain, and GIoU represents the generalized intersection loss on the union.

[0072] Then, the binary cross-entropy (BCE) loss can be used as the center-aware loss function. Specifically, it can be calculated according to the formula... The center perception pseudo-label and the center perception prediction results of the target domain (which can be represented as) ) is calculated to obtain the target domain center perceptual loss (which can be expressed as ).in, This represents the loss of perception at the center of the target domain. y represents the predicted result of the center perception of the target domain. t This represents a target domain pseudo-label, which includes a center perception pseudo-label.

[0073] Finally, according to the formula The target domain category loss, target domain regression loss, and target domain center perception loss are calculated to obtain the target domain loss after supervised training. Where x T Represents the target domain image, y t Indicates a pseudo-tag for the target domain. This represents the target domain loss between the target domain detection result and the target domain pseudo-label. Indicates the target domain category loss. This represents the regression loss in the target domain. This represents the loss of perception at the center of the target domain. This indicates the target domain category prediction result. This indicates the regression prediction results for the target domain. This represents the predicted result of the center perception of the target domain.

[0074] Understandably, although the network structures of the teacher and student models are similar, the student model's parameter updates use the traditional gradient descent-based method. During training, the student network receives supervised learning from real labeled data from the source domain and pseudo-labels generated by the teacher network for unlabeled target domain data. Its parameters are then updated through backpropagation and optimization algorithms (such as SGD and Adam). However, the teacher model's weight parameters are updated differently. The teacher model's weight parameters are updated using Exponential Moving Average (EMA). This means that the teacher network's parameters are not directly optimized based on gradient backpropagation, but rather updated periodically or according to a predetermined strategy by smoothing the student network's parameters at a certain decay rate. This method allows the teacher network to maintain a more stable and generalized feature representation, thus generating relatively high-quality pseudo-labels. The aforementioned EMA, also called Weighted Moving Average, is an averaging method that assigns higher weights to recent data, specifically expressed by the formula θ′. t =αθ′ t-1 +(1-α)θ s Where α is the smoothing coefficient hyperparameter, and θ′ t θ′ represents the weight parameters of the teacher model in this round. t-1 θ represents the weight parameters of the previous teacher model. s These are the weight parameters of the student model.

[0075] As an optional implementation of step S150 above, the implementation of calculating the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset may include:

[0076] Step S151: Obtain a random mask and channel variables. The size of the random mask is the same as the size of the multi-scale feature map of the target domain, and the number of channels of the channel variables is the same as the number of channels of the multi-scale feature map of the target domain.

[0077] The process of generating the random mask described above is as follows: A random vector with a value of 1 is generated, with the same size as the input features. Then, the number of retained features is calculated, which is the scale information of the input fused features (including batch size, channel size, length, and width). Specifically, the number of retained features is calculated using the formula: Number of retained features = Length * Width * (1 - Masking rate). Then, the number of retained features is set to 0, meaning that in the random mask vector, 0 indicates that the original feature values ​​are retained, and 1 indicates that the original feature values ​​are masked. Random noise is generated here to retain some features; this random noise is used to determine which features need to be retained and which need to be replaced. Finally, the vector is sorted, and the sorted index values ​​are extracted. Then, the obtained index values ​​are used to sort the random mask vector.

[0078] The process of generating the channel variables described above is as follows: a learnable channel vector is generated, corresponding to each channel of the input feature. The function of this channel is similar to the channel attention principle, which reduces the randomness of the mask by learning the importance of the feature channels.

[0079] Step S152: Calculate the random mask, channel variables, and source domain multi-scale feature map using the multi-scale mask adversarial alignment module to obtain source domain reconstructed features.

[0080] Understandably, the essence of the aforementioned feature reconstruction is to sparse the feature distribution. When the model makes a mistake in the pseudo-label, i.e., a positive sample is classified as a negative sample, the model's decision boundary can be corrected by reconstructing the features, forcing the model to re-determine the decision boundary. When adversarial training domain adaptation is added, the backbone network in the student model can extract domain-invariant features and input these features into the detection head network, causing the model to consider the domain-invariant features as positive samples. These domain-invariant features are the foreground region features in the source and target domains.

[0081] Step S153: Calculate the random mask, channel variables, and target domain multi-scale feature map using the multi-scale mask adversarial alignment module to obtain the target domain reconstruction features.

[0082] Step S154: Calculate the source domain reconstruction features and target domain reconstruction features to obtain the alignment loss.

[0083] Specifically, multi-scale reconstructed features can be input into a gradient flipping layer and a domain discriminator. By using an adversarial discrimination method, pseudo-labels 1 and 0 are assigned to the source and target domains. Then, a loss function is used to calculate the alignment loss. The specific calculation process will be described in detail below.

[0084] In the implementation of the above scheme, the target domain lacks labeled information, and interference from the background region easily leads to misclassification by the model. This causes the foreground region of the target domain image to be easily classified as a negative sample. The backbone network and feature fusion layer in the student model act as feature extractors. During domain-adaptive training, these feature extractors often focus more on source domain data features, thus classifying the foreground region of the target domain image as background features and ignoring foreground information. Therefore, by reconstructing features from both the source and target domains, the student model can pay more attention to the features of the foreground region of the target domain image, thereby improving the detection accuracy of the student model for the target domain image.

[0085] As an optional implementation of step S152 above, the multi-scale mask adversarial alignment module includes: a fully convolutional decoder; the implementation of calculating random masks, channel variables, and source domain multi-scale feature maps through the multi-scale mask adversarial alignment module may include:

[0086] Step S152a: Apply mask weighting to the random mask, channel variables, and source domain multi-scale feature map to obtain the mask-weighted source domain scale features.

[0087] An example of the implementation of step S152a above is as follows: the multi-scale mask adversarial alignment module described above can be implemented using the formula... Mask weighting is applied to the random mask, channel variables, and source domain multi-scale feature maps to obtain the mask-weighted source domain scale features. Where M... mask Let m represent the source domain scale features after mask weighting, M represent the source domain multi-scale feature map output by the feature fusion layer (e.g., feature pyramid) in the student model, m(μ) represent the random mask vector corresponding to the masking ratio μ, that is, in this vector, the pixel value of μ is 0, and the value of the remaining pixels is 1. The size of m is consistent with the resolution of the input features, and v is a learnable channel variable (also known as the channel vector), whose size is consistent with the number of channels of the input features.

[0088] Step S152b: Use a fully convolutional decoder to perform mask decoding on the mask-weighted source domain scale features to obtain the source domain reconstructed features.

[0089] For example, in implementing step S152b above: the input features after masking become sparse, so these features are input into a fully convolutional decoder. That is, the fully convolutional decoder is used to perform mask decoding on each layer of mask features in the mask-weighted source domain scale features to obtain source domain reconstructed features, which can be understood as source domain multi-scale reconstructed features f. mask .

[0090] As an optional implementation of step S153 above, the multi-scale mask adversarial alignment module includes: a fully convolutional decoder; the implementation of calculating random masks, channel variables, and multi-scale feature maps of the target domain through the multi-scale mask adversarial alignment module may include:

[0091] Step S153a: Apply mask weighting to the random mask, channel variables, and target domain multi-scale feature map to obtain the mask-weighted multi-scale features.

[0092] An example of the implementation of step S153a above is as follows: the multi-scale mask adversarial alignment module described above can be implemented using the formula... Mask weighting is applied to the random mask, channel variables, and multi-scale feature maps of the target domain to obtain the mask-weighted multi-scale features. Where M... mask Let M represent the multi-scale feature after mask weighting, M represent the target domain multi-scale feature map output by the feature fusion layer (e.g., feature pyramid) in the student model, m(μ) represent the random mask vector corresponding to the masking ratio μ, that is, in this vector, the pixel value of μ is 0, and the value of the remaining pixels is 1. The size of m is consistent with the resolution of the input feature, and v is the learnable channel variable (also known as the channel vector), whose size is consistent with the number of channels of the input feature.

[0093] Step S153b: Use a fully convolutional decoder to perform mask decoding on the mask-weighted multi-scale features to obtain the target domain reconstructed features.

[0094] For example, in the implementation of step S153b above: the input features after masking become sparse, so these features are input into a fully convolutional decoder. That is, the fully convolutional decoder is used to perform mask decoding on each layer of mask features in the mask-weighted target domain scale features to obtain the target domain reconstruction features, which can be understood as the target domain multi-scale reconstruction features f. mask .

[0095] As an optional implementation of step S154 above, the multi-scale mask adversarial alignment module further includes: a gradient flip layer and a domain discriminator; calculating the source domain reconstruction features and the target domain reconstruction features, including:

[0096] Step S154a: Use a gradient flipping layer to align the source domain reconstruction features and the target domain reconstruction features to obtain aligned source domain reconstruction features and aligned target domain reconstruction features.

[0097] Since both the teacher and student models can be single-stage detection networks, the positive and negative samples of single-stage detection networks are usually imbalanced. Furthermore, the teacher model of a single-stage detection network cannot filter out all low-quality pseudo-labels, and the student model of a single-stage detection network is very sensitive to low-quality pseudo-labels. These low-quality pseudo-labels can easily cause the student model to fall into suboptimal results.

[0098] Step S154b: Use the domain discriminator to calculate the alignment loss between the aligned source domain reconstruction features and the aligned target domain reconstruction features.

[0099] For example, the implementation of steps S154a to S154b above involves using a gradient flipping layer to align the source domain reconstructed features and the target domain reconstructed features, obtaining aligned source domain reconstructed features and aligned target domain reconstructed features. The domain discriminator then uses the following formula...

[0100]

[0101] The alignment loss between the aligned source domain reconstructed features and the aligned target domain reconstructed features is calculated. This represents the reconstructed features of the source domain after alignment. This represents the reconstructed features of the aligned target domain. N represents the alignment loss between the aligned source domain reconstructed features and the aligned target domain reconstructed features. S and N T γ represents the number of source domain images in the source domain dataset and the number of target domain images in the target domain dataset, respectively. D represents the domain discriminator, and γ is the confidence factor, which is used to control the magnitude of the loss weights.

[0102] In the implementation of the above scheme, random mask weighting and decoding are used in the multi-scale mask adversarial alignment module to reconstruct multi-scale features. Combined with adversarial training methods, the student model is helped to escape the local suboptimal results caused by erroneous pseudo-labels, thereby achieving the effect of pixel-level feature alignment and ultimately improving the detection accuracy of the student model for target domain images.

[0103] As an optional implementation of step S160 above, since the overall network optimization objective is divided into supervised source domain loss, unsupervised target domain loss, and multi-scale mask adversarial alignment loss, supervised training of only the source domain is performed in the early stages of student model training. This aims to avoid the teacher model accumulating erroneous decisions made by the student model in the early stages. Therefore, the following formula can be used:

[0104]

[0105] The source domain loss, target domain loss, and alignment loss are calculated to obtain the total loss value. This total loss value is then used to train the student model, resulting in the trained student model. This represents the total loss value. x represents the source domain loss between the source domain detection result and the source domain label. S Represents the source domain image, y s Indicates the source domain tag. x represents the target domain loss between the target domain detection result and the target domain pseudo-label. T Represents the target domain image, y t Indicates a pseudo-tag for the target domain. This represents the alignment loss between the aligned source domain reconstructed features and the aligned target domain reconstructed features. This represents the reconstructed features of the source domain after alignment. The target domain reconstructed features are represented by the aligned features, where i and I are the current training iteration number and the pre-training end iteration threshold, respectively, and δ is a hyperparameter used to balance the overall loss.

[0106] Please see Figure 3 The illustration shows a flowchart of a cross-domain target detection method provided in an embodiment of this application; this application provides a cross-domain target detection method, including:

[0107] Step S210: Obtain the image to be processed.

[0108] The implementation methods of step S210 above include: First, using a terminal device such as a camera, video recorder, or color camera to capture an image of the target object to obtain an image to be processed; then, the terminal device sends the image to be processed to an electronic device, and the electronic device receives the image to be processed sent by the terminal device. The electronic device can store the image to be processed in a file system, database, or mobile storage device. Second, obtaining a pre-stored image to be processed, specifically, for example, obtaining the image to be processed from a file system, database, or mobile storage device. Third, using software such as a browser to obtain the image to be processed from the Internet, or using other applications to access the Internet to obtain the image to be processed.

[0109] Step S220: Use the trained student model to perform object detection on the image to be processed.

[0110] For example, in implementing step S220 above, after obtaining the trained student model, the target domain detection head network and the multi-scale mask adversarial alignment module in the trained student model can be pruned to obtain a pruned student model. The pruned student model is then used to perform target detection on the image to be processed. The resulting target detection result can include: predicted category and predicted bounding box. In the above scheme, by pruning the target domain detection head network and the multi-scale mask adversarial alignment module in the trained student model, the student model only needs to use the backbone network, feature fusion layer, and source domain detection head network to complete target detection on the image to be processed during inference, thereby reducing additional computational costs and effectively improving the efficiency of cross-domain target detection.

[0111] Please see Figure 4 The diagram shown is a flowchart of a model training apparatus provided in an embodiment of this application; this application provides a model training apparatus 300, including:

[0112] The domain dataset acquisition module 310 is used to acquire a labeled source domain dataset and an unlabeled target domain dataset. The target domain dataset includes: target domain images.

[0113] The source domain loss calculation module 320 is used to calculate the source domain loss of the source domain detection head network in the student model based on the source domain dataset.

[0114] The domain pseudo-label acquisition module 330 is used to perform target detection on the target domain image through the teacher model and obtain the target domain pseudo-label. The weight parameters of the teacher model are obtained by assigning values ​​to the weight parameters of the student model.

[0115] The target loss calculation module 340 is used to calculate the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-label and the target domain image.

[0116] Alignment loss calculation module 350 is used to calculate the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset.

[0117] The student model training module 360 ​​is used to train the student model based on the source domain loss, target domain loss, and alignment loss to obtain the trained student model.

[0118] As an optional implementation of the above-described apparatus, the source domain dataset includes: source domain images and source domain labels; the source domain loss calculation module includes:

[0119] The source domain feature extraction submodule is used to extract features from the source domain image through the backbone network in the student model to obtain a multi-layer feature map of the source domain.

[0120] The source domain feature fusion submodule is used to perform multi-scale feature fusion on the multi-layer feature map of the source domain through the feature fusion layer in the student model to obtain the multi-scale feature map of the source domain.

[0121] The source domain target detection submodule is used to perform target detection on the source domain multi-scale feature map through the source domain detection head network in the student model to obtain the source domain detection results.

[0122] The source domain loss determination submodule is used to determine the source domain loss as the loss value between the source domain detection result and the source domain label.

[0123] As an optional implementation of the above-mentioned device, the target loss calculation module includes:

[0124] The target domain feature extraction submodule is used to extract features from the target domain image through the backbone network in the student model to obtain a multi-layer feature map of the target domain.

[0125] The target domain feature fusion submodule is used to perform multi-scale feature fusion on the multi-layer feature map of the target domain through the feature fusion layer in the student model to obtain the multi-scale feature map of the target domain.

[0126] The target domain target detection submodule is used to perform target detection on the multi-scale feature map of the target domain through the target domain detection head network in the student model, and obtain the target domain detection result.

[0127] The target domain loss determination submodule is used to determine the target domain loss by comparing the target domain detection result with the target domain pseudo-label.

[0128] As an optional implementation of the above-mentioned device, the alignment loss calculation module includes:

[0129] The mask variable acquisition submodule is used to acquire random masks and channel variables. The size of the random mask is the same as the size of the multi-scale feature map of the target domain, and the number of channels of the channel variables is the same as the number of channels of the multi-scale feature map of the target domain.

[0130] The reconstructed feature acquisition submodule is used to calculate the source domain reconstructed features by using the multi-scale mask adversarial alignment module to calculate the random mask, channel variables, and source domain multi-scale feature map.

[0131] The scale feature acquisition submodule is used to calculate the target domain reconstruction features by using the multi-scale mask adversarial alignment module to calculate the random mask, channel variables and target domain multi-scale feature map.

[0132] The alignment loss acquisition submodule is used to calculate the alignment loss by analyzing the source domain reconstruction features and the target domain reconstruction features.

[0133] As an optional implementation of the above-mentioned device, the multi-scale mask adversarial alignment module includes: a fully convolutional decoder; and a scale feature acquisition submodule, including:

[0134] The feature mask weighting unit is used to perform mask weighting on random masks, channel variables, and multi-scale feature maps of the target domain to obtain mask-weighted multi-scale features.

[0135] The feature mask decoding unit is used to perform mask decoding on the mask-weighted multi-scale features using a fully convolutional decoder to obtain the reconstructed features of the target domain.

[0136] As an optional implementation of the above-mentioned device, the multi-scale mask adversarial alignment module further includes: a gradient flip layer and a domain discriminator; the alignment loss acquisition submodule includes:

[0137] The domain feature alignment unit is used to align the source domain reconstructed features and the target domain reconstructed features using a gradient flipping layer, so as to obtain the aligned source domain reconstructed features and the aligned target domain reconstructed features.

[0138] The alignment loss calculation unit is used to calculate the alignment loss between the aligned source domain reconstruction features and the aligned target domain reconstruction features using a domain discriminator.

[0139] This application also provides a cross-domain target detection device, including:

[0140] The image acquisition module is used to acquire images to be processed.

[0141] The image object detection module is used to perform object detection on the image to be processed using the student model described above.

[0142] It should be understood that this device corresponds to the model training method and cross-domain object detection method embodiments described above, and is capable of performing the various steps involved in the above method embodiments. The specific functions of this device can be found in the description above, and detailed descriptions are appropriately omitted here. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0143] Please see Figure 5 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 400 provided in this application includes a processor 410 and a memory 420. The memory 420 stores machine-readable instructions executable by the processor 410. When the machine-readable instructions are executed by the processor 410, the method described above is performed.

[0144] This application embodiment also provides a computer-readable storage medium 430, on which a computer program is stored. This computer program is executed by a processor 410 to perform the methods described above. The computer-readable storage medium 430 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0145] This application also provides a computer program product, including: a computer program or computer instructions, which are executed by a processor to perform the method described above.

[0146] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0147] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending primarily on the functions involved.

[0148] Furthermore, the functional modules of each embodiment in this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," "some examples," etc., means that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0149] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A model training method, characterized in that, include: Obtain a labeled source domain dataset and an unlabeled target domain dataset, wherein the target domain dataset includes: target domain images; The source domain loss of the source domain detection head network in the student model is calculated based on the source domain dataset. The target domain image is detected by the teacher model to obtain the target domain pseudo label. The weight parameters of the teacher model are obtained by assigning values ​​to the weight parameters of the student model. The target domain loss of the target domain detection head network in the student model is calculated based on the target domain pseudo-label and the target domain image. The alignment loss of the multi-scale mask adversarial alignment module in the student model is calculated based on the source domain dataset and the target domain dataset. The student model is trained based on the source domain loss, the target domain loss, and the alignment loss to obtain a trained student model, which is used for object detection in an image. The step of calculating the alignment loss of the multi-scale mask adversarial alignment module in the student model based on the source domain dataset and the target domain dataset includes: Obtain a random mask and channel variables, wherein the size of the random mask is the same as the size of the multi-scale feature map of the target domain, and the number of channels of the channel variables is the same as the number of channels of the multi-scale feature map of the target domain; The source domain reconstruction features are obtained by calculating the random mask, the channel variables, and the source domain multi-scale feature map through the multi-scale mask adversarial alignment module. The multi-scale mask adversarial alignment module calculates the random mask, the channel variables, and the multi-scale feature map of the target domain to obtain the target domain reconstruction features. The alignment loss is obtained by calculating the source domain reconstruction features and the target domain reconstruction features.

2. The method according to claim 1, characterized in that, The source domain dataset includes: source domain images and source domain labels; the calculation of the source domain loss of the source domain detection head network in the student model based on the source domain dataset includes: Feature extraction of the source domain image is performed using the backbone network in the student model to obtain a multi-layer feature map of the source domain; The source domain multi-scale feature map is obtained by performing multi-scale feature fusion on the source domain multi-layer feature map through the feature fusion layer in the student model. The source domain detection head network in the student model is used to perform target detection on the source domain multi-scale feature map to obtain the source domain detection result. The loss value between the source domain detection result and the source domain label is determined as the source domain loss.

3. The method according to claim 2, characterized in that, The step of calculating the target domain loss of the target domain detection head network in the student model based on the target domain pseudo-label and the target domain image includes: Feature extraction of the target domain image is performed using the backbone network in the student model to obtain a multi-layer feature map of the target domain; The target domain multi-scale feature map is obtained by performing multi-scale feature fusion on the multi-layer feature map of the target domain through the feature fusion layer in the student model. The target domain detection head network in the student model is used to perform target detection on the multi-scale feature map of the target domain to obtain the target domain detection result. The loss value between the target domain detection result and the target domain pseudo-label is determined as the target domain loss.

4. The method according to claim 1, characterized in that, The multi-scale mask adversarial alignment module includes a fully convolutional decoder; the calculation of the random mask, the channel variables, and the target domain multi-scale feature map through the multi-scale mask adversarial alignment module includes: The random mask, the channel variables, and the target domain multi-scale feature map are mask-weighted to obtain the mask-weighted multi-scale features; The fully convolutional decoder is used to perform mask decoding on the mask-weighted multi-scale features to obtain the target domain reconstructed features.

5. The method according to claim 4, characterized in that, The multi-scale mask adversarial alignment module further includes: a gradient flip layer and a domain discriminator; the calculation of the source domain reconstruction features and the target domain reconstruction features includes: The gradient flipping layer is used to align the source domain reconstruction features and the target domain reconstruction features to obtain aligned source domain reconstruction features and aligned target domain reconstruction features. The alignment loss between the aligned source domain reconstruction features and the aligned target domain reconstruction features is calculated using the domain discriminator.

6. A cross-domain target detection method, characterized in that, include: Obtain the image to be processed; The trained student model according to any one of claims 1 to 5 is used to perform object detection on the image to be processed.

7. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, the machine-readable instructions being executed by the processor to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that, include: A computer program or computer instructions that, when executed by a processor, perform the method according to any one of claims 1 to 6.

Images