Training method and device of cross-domain object detection model for image and cross-domain object detection method

Abstract

The application discloses a training method and device of a cross-domain target detection model for images and a cross-domain target detection method. The training method comprises the following steps: acquiring source domain data and target domain data; inputting the source domain data and the target domain data into a deep learning network to obtain a first feature map; adjusting attention weights in the first feature map based on a cross-channel cross-attention strategy to obtain a second feature map with adjusted attention weights; inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific class feature alignment to obtain a third feature map; inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; and adjusting model parameters according to the source domain data and the target domain data and the first to fourth feature maps to obtain a trained cross-domain target detection model.

Description

Technical Field

[0001] This disclosure generally relates to the field of computer vision technology, and more specifically, to a training method and apparatus for a cross-domain object detection model for images, and a cross-domain object detection method. Background Technology

[0002] In related fields, with the rapid development of computer vision technology, object detection has become an important research direction. The purpose of object detection is to automatically identify objects in images or videos and determine their location and category. Object detection technology has a wide range of applications, such as intelligent transportation systems, video surveillance, and autonomous driving.

[0003] To achieve object detection, common methods include image processing and machine learning. With the development of machine learning technology, the accuracy and efficiency of object detection have continuously improved, but a series of problems have also emerged. For example, the training dataset used to train the model typically requires a sufficient amount of labeled data to ensure high accuracy. However, manual labeling consumes significant human and material resources, and due to environmental factors such as lighting and weather conditions, the data in real-world applications is far more complex and variable than the collected datasets. Furthermore, when transferring models between different scenarios, significant differences in domain distribution can lead to a substantial drop in model performance. Summary of the Invention

[0004] The embodiments of this disclosure provide a training method and apparatus for a cross-domain object detection model for images, and a cross-domain object detection method. By constructing a cross-domain strategy that integrates adversarial learning and self-aware hierarchical alignment, a cross-domain weakly supervised object detection algorithm for image data is realized, thereby improving the accuracy of image detection tasks in cross-domain scenarios.

[0005] In one general aspect, a training method for a cross-domain object detection model for images is provided. The training method includes: acquiring source domain data and target domain data, wherein the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data; inputting the source domain data and target domain data into a pre-trained deep learning network to obtain a first feature map for both the source and target domain images; adjusting the attention weights in the first feature map based on a cross-channel attention strategy to obtain adjusted attention weights, and using the first feature map with the adjusted attention weights as a second feature map; inputting the second feature map into a system for... A domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment are used to obtain a third feature map after feature alignment. The third feature map is then input into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map. Based on the source domain data, the target domain data, and the first to fourth feature maps, the model parameters of the cross-domain object detection model are adjusted to obtain a trained cross-domain object detection model. The cross-domain object detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network, and the second prediction network.

[0006] Optionally, the step of adjusting the attention weights in the first feature map based on the cross-channel cross-attention strategy to obtain the adjusted attention weights may include: for each feature included in the deep and shallow features of the first feature map, adjusting the attention weight of each channel according to the following formula to obtain the adjusted attention weights:

[0007] g=C1D {k} (g(χ))

[0008] Where g represents the attention weight, C1D represents fast one-dimensional convolution, and k represents the number of neighboring channels, and the expression for k is: g(χ) represents the global average pooling of the channel dimension of the image data, and the expression for g(χ) is: Where C represents the channel dimension, γ and b are preset parameter values, | | odd Represents the nearest odd number. The tensor data represents the image data. W and H represent the width and height of the tensor, respectively. i is a natural number ranging from 1 to the value of W, and j is a natural number ranging from 1 to the value of H.

[0009] Optionally, the step of inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for class-specific feature alignment to obtain a feature-aligned third feature map may include: adding a domain label to each feature in the second feature map, the domain label indicating the correspondence between each feature and the source and target domains; after performing the following processing in parallel, combining all features obtained after performing image-level feature alignment on both deep and shallow features in the second feature map and class-specific feature alignment on the deep features in the second feature map to obtain the third feature map. Feature Mapping: The second feature map is input into a global domain classifier for deep features and a local domain classifier for shallow features. The global and local domain classifiers are trained using an adversarial learning method, and the training parameters are adjusted using a domain classification loss function to achieve image-level feature alignment between source and target domain data. Additionally, the deep features included in the first feature map are input into the image-level multi-label classifier network. The image-level multi-label classifier network is trained using a supervised learning method, and the training parameters are adjusted using an image-level classification loss function to achieve specific category feature alignment between source and target domain data.

[0010] Optionally, the step of inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map may include: after performing the following processes in parallel, combining all features after performing instance-level feature alignment on features in the third feature map associated with source domain data and performing image-level category prediction-based instance-level feature alignment on features in the third feature map associated with target domain data to obtain the fourth feature map: inputting the third feature map and the source domain data into the first prediction network for source domain instance-level feature alignment, training the first prediction network using a supervised learning method to achieve instance-level feature alignment of the source domain data; inputting the third feature map and the target domain data into the second prediction network for target domain instance-level category prediction, training the second prediction network using a weakly supervised learning method to achieve image-level category prediction-based instance-level feature alignment of the target domain data.

[0011] Optionally, the source domain image annotation data may include first image-level annotation information and instance-level annotation information, and the step of inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of the source domain data may include: inputting the third feature map, the first image-level annotation information, and the instance-level annotation information into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information; calculating a first loss function based on the first image-level annotation information, the instance-level annotation information, the first prediction information, and the second prediction information, and adjusting the network parameters of the first prediction network according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0012] Optionally, the target domain image annotation data may include second image-level annotation information, and the step of inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network using a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction may include: inputting deep features in the third feature map and the second image-level annotation information into the second prediction network to obtain third prediction information and fourth prediction information, wherein the third prediction information represents the confidence score of a candidate box associated with the deep features containing a target object, and the fourth prediction information represents the prediction score of the target object in the candidate box belonging to a specific category; reconstructing the instance-level specific category prediction score based on the third prediction information and the fourth prediction information; obtaining the final prediction information based on image-level category prediction based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information; calculating a second loss function based on the second image-level annotation information and the final prediction information, and adjusting the network parameters of the second prediction network according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0013] In another general aspect, a training apparatus for a cross-domain object detection model for images is provided, the training apparatus comprising: a data acquisition module configured to acquire source domain data and target domain data, the source domain data including source domain image data and source domain image annotation data, and the target domain data including target domain image data and target domain image annotation data; a feature extraction module configured to input the source domain data and target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; a weight adjustment module configured to adjust the attention weights in the first feature map based on a cross-channel cross-attention strategy to obtain adjusted attention weights, and use the first feature map with the adjusted attention weights as a second feature map; and a first feature alignment module configured to: The second feature map is input to a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment; the second feature alignment module is configured to input the third feature map to a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; the parameter adjustment module is configured to adjust the model parameters of the cross-domain object detection model according to the source domain data, the target domain data, and the first to fourth feature maps to obtain a trained cross-domain object detection model, wherein the cross-domain object detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network, and the second prediction network.

[0014] Optionally, the weight adjustment module adjusts the attention weights in the first feature map based on a cross-channel attention strategy to obtain the adjusted attention weights. This operation may include: for each feature included in the deep and shallow features of the first feature map, adjusting the attention weight of each channel according to the following formula to obtain the adjusted attention weights:

[0015] g=C1D {k} (g(χ))

[0016] Where g represents the attention weight, C1D represents fast one-dimensional convolution, and k represents the number of neighboring channels, and the expression for k is: g(χ) represents the global average pooling of the channel dimension of the image data, and the expression for g(χ) is: Where C represents the channel dimension, γ and b are preset parameter values, || odd Represents the nearest odd number. The tensor data represents the image data. W and H represent the width and height of the tensor, respectively. i is a natural number ranging from 1 to the value of W, and j is a natural number ranging from 1 to the value of H.

[0017] Optionally, the first feature alignment module inputs the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a feature-aligned third feature map. This operation may include: adding a domain label to each feature in the second feature map, the domain label indicating the correspondence between each feature and the source and target domains; after performing the following processes in parallel, combining all features obtained after performing image-level feature alignment on both deep and shallow features in the second feature map and specific category feature alignment on the deep features in the second feature map, to obtain the final feature map. The third feature map is described as follows: The second feature map is input into a global domain classifier for deep features and a local domain classifier for shallow features. The global domain classifier and the local domain classifier are trained using an adversarial learning method, and the training parameters are adjusted using a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data. Furthermore, the deep features included in the first feature map are input into the image-level multi-label classifier network. The image-level multi-label classifier network is trained using a supervised learning method, and the training parameters are adjusted using an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0018] Optionally, the operation of the second feature alignment module to input the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map may include: after performing the following processes in parallel, combining all features after performing instance-level feature alignment on features in the third feature map associated with source domain data and performing image-level category prediction-based instance-level feature alignment on features in the third feature map associated with target domain data to obtain the fourth feature map: inputting the third feature map and the source domain data into the first prediction network for source domain instance-level feature alignment, training the first prediction network using a supervised learning method to achieve instance-level feature alignment of the source domain data; inputting the third feature map and the target domain data into the second prediction network for target domain instance-level category prediction, training the second prediction network using a weakly supervised learning method to achieve image-level category prediction-based instance-level feature alignment of the target domain data.

[0019] Optionally, the source domain image annotation data may include first image-level annotation information and instance-level annotation information. The second feature alignment module inputs the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment. The operation of training the first prediction network through a supervised learning method to achieve instance-level feature alignment of the source domain data may include: inputting the third feature map, the first image-level annotation information, and the instance-level annotation information into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information; calculating a first loss function based on the first image-level annotation information, the instance-level annotation information, the first prediction information, and the second prediction information; and adjusting the network parameters of the first prediction network according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0020] Optionally, the target domain image annotation data may include second image-level annotation information, and the second feature alignment module inputs the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction. The second prediction network is trained using a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction. This operation may include: inputting deep features from the third feature map and the second image-level annotation information into the second prediction network to obtain third and fourth prediction information. The third prediction information represents the confidence score of a candidate box associated with the deep features containing a target object, and the fourth prediction information represents the prediction score of the target object in the candidate box belonging to a specific category; reconstructing the instance-level specific category prediction score based on the third and fourth prediction information; obtaining final prediction information based on image-level category prediction based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information; calculating a second loss function based on the second image-level annotation information and the final prediction information, and adjusting the network parameters of the second prediction network according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0021] In another general aspect, a cross-domain object detection method for image data is provided, the cross-domain object detection method comprising: acquiring image data to be detected; inputting the image data to be detected into a cross-domain object detection model for images to obtain corresponding image detection data, wherein the cross-domain object detection model is trained using the training method described above.

[0022] In another general aspect, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the training method for a cross-domain object detection model for images and the cross-domain object detection method for image data as described above.

[0023] In another general aspect, a computer-readable storage medium is provided that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, enables the electronic device / server to perform the training method for a cross-domain object detection model for an image and the cross-domain object detection method for image data as described above.

[0024] In another general aspect, a computing device is provided, the computing device comprising: at least one processor; at least one memory storing computer-executable instructions, wherein, when executed by the at least one processor, the computer-executable instructions cause the at least one processor to perform the training method for a cross-domain object detection model for an image and the cross-domain object detection method for image data as described above.

[0025] The training method and apparatus for cross-domain object detection models for images, and the cross-domain object detection method according to embodiments of this disclosure, realize a weakly supervised cross-domain object detection algorithm for image data by constructing a cross-domain strategy that integrates adversarial learning and self-aware hierarchical alignment, thereby improving the accuracy of image detection tasks in cross-domain scenarios. Furthermore, the training method and apparatus for cross-domain object detection models for images, and the cross-domain object detection method according to embodiments of this disclosure address the problem of significant performance degradation caused by domain differences. Attached Figure Description

[0026] The above and other objects and features of the embodiments of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings illustrating the embodiments, wherein:

[0027] Figure 1 This is a flowchart illustrating a training method for a cross-domain object detection model for images according to an embodiment of the present disclosure;

[0028] Figure 2 This is an example structural diagram illustrating a channel interaction attention network according to this disclosure;

[0029] Figure 3 This is an example structural diagram illustrating an instance-level weakly supervised network according to an embodiment of the present disclosure;

[0030] Figure 4 This is an example flowchart illustrating an instance-level weak supervision method according to an embodiment of the present disclosure;

[0031] Figure 5 This is a schematic diagram illustrating an exemplary network structure according to an embodiment of the present disclosure;

[0032] Figure 6 This is a block diagram illustrating a training apparatus for a cross-domain object detection model for images according to an embodiment of the present disclosure;

[0033] Figure 7 This is a block diagram illustrating a computing device according to an embodiment of the present disclosure. Detailed Implementation

[0034] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0035] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, examples of which are illustrated in the drawings, wherein the same reference numerals always refer to the same parts. The embodiments will now be described with reference to the accompanying drawings in order to explain this disclosure.

[0036] As mentioned above, existing neural network-based object detection methods often require sufficiently labeled datasets to achieve higher accuracy. However, real-world data is far more complex and varied than collected datasets, leading to significant performance degradation during model transfer due to large differences in domain distribution. Therefore, weakly supervised domain adaptation methods that utilize additional image-level annotations to transfer information learned on labeled datasets to new domains have gained widespread attention across various fields.

[0037] The learning scenario based on weakly supervised domain adaptation limits the dataset with a large number of labeled samples to the source domain, while the dataset with no or only a small number of labeled samples is used as the target domain. Currently, the main domain adaptation methods include the following two types: domain adaptation classification based on adversarial learning and domain adaptation based on self-training.

[0038] Specifically, for the former, a typical network is the Domain Adaptation Neural Network (DANN) based on adversarial learning. For the latter, this type of method assists training by generating high-quality pseudo-labels on the target domain data to fully utilize the effective information of the target domain. It is highly dependent on the quality of pseudo-label generation. Typical methods include the Unbiased Mean Teacher (UMT) method and the Adaptive Teacher (AT) method based on teacher-student distillation models.

[0039] However, these methods may have a series of problems, such as the inability to accurately capture complex distribution differences of local features, difficulty in aligning local domains, and poor model generalization.

[0040] This disclosure proposes a training method and apparatus for a cross-domain object detection model for images, as well as a cross-domain object detection method. By adopting adversarial learning and self-aware hierarchical alignment strategies, a weakly supervised object detection algorithm for images across domains is realized, which can improve the accuracy of object detection tasks in cross-domain scenarios and effectively solve the problem of significant decline in model detection performance caused by domain differences.

[0041] The following reference Figures 1 to 7 This document provides a detailed description of a training method and apparatus for a cross-domain object detection model for images, and a cross-domain object detection method, according to embodiments of the present disclosure.

[0042] First, refer to Figures 1 to 5 This document describes a training method for a cross-domain object detection model for images according to embodiments of the present disclosure.

[0043] Figure 1 This is a flowchart illustrating a training method 100 for a cross-domain object detection model for an image according to an embodiment of the present disclosure. Figure 2 This is an example structural diagram illustrating a channel interaction attention network according to this disclosure. Figure 3 This is an example structural diagram illustrating an instance-level weakly supervised network according to an embodiment of the present disclosure.

[0044] Figure 4 This is an example flowchart illustrating an instance-level weak supervision method according to an embodiment of the present disclosure. Figure 5 This is a schematic diagram illustrating an exemplary network structure according to an embodiment of the present disclosure.

[0045] As an example, a cross-domain object detection model for images includes a deep learning network, a domain classifier network, an image-level multi-label classifier network, a first prediction network, and a second prediction network.

[0046] Reference Figure 1According to an embodiment of this disclosure, in step S101, source domain data and target domain data are acquired. Here, the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data.

[0047] Furthermore, the source domain image annotation data includes image-level annotation information (i.e., category annotation information) and instance-level annotation information (i.e., annotation information corresponding to bounding box regression). The target domain image annotation data includes image-level annotation information, but does not include or includes very little instance-level annotation information. In addition, the image-level annotation information included in the source domain can also be referred to as the first image-level annotation information, and the image-level annotation information included in the target domain can also be referred to as the second image-level annotation information.

[0048] According to an embodiment of this disclosure, in step S102, source domain data and target domain data are input into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image.

[0049] For example, the first feature map may consist only of deep feature maps, or it may include both deep and shallow feature maps. That is, as an example, deep features can be output as the output of a deep learning network to the next network, while shallow features can be extracted when other networks need them. As another example, a deep learning network may also output a feature map that includes both deep features (also known as global features) and shallow features (also known as local features).

[0050] In other words, image feature extraction of the source and target domains can be achieved through step S102.

[0051] Specifically, firstly, the source domain image data (hereinafter referred to as source domain image) and target domain image data (hereinafter referred to as target domain image) are represented as numerical tensors; then, pre-trained backbone networks (e.g., pre-trained models in deep learning, such as but not limited to VGG16, ResNet101, Inception-v3, etc.) are used. Figure 5 The schematic Backbone shown in the diagram performs convolution operations on the image tensor to extract image features.

[0052] For example, a specific example of step S102 can be as follows:

[0053] First, the input source and target domain images can be represented as numerical tensors [C, W, H], where C represents the three channels of the image pixels, and W and H represent the width and height of the tensor, respectively. Image tensors are then selected from the source domain S and the target domain T and combined to obtain the model input tensor representation. The specific expression is shown in equation (1) below:

[0054] X=(Xt,Xs)=[2N,C,W,H] (1)

[0055] Where Xt and Xs represent multidimensional tensors from the target domain and source domain, respectively, and 2N represents the total batch size of the input images, with the first N images from the target domain and the last N images from the source domain.

[0056] Then, the input image tensor is processed by the convolutional layer, batch normalization layer and ReLU activation function of the pre-trained backbone network to extract features, and the corresponding feature representation F is obtained. The specific expression is shown in the following equation (2):

[0057] F = BackBone(Xt, Xs) (2)

[0058] The step of extracting image features by feeding image data into a deep convolutional neural network can extract high-level semantic features of the image through multiple stacked convolutional layers, batch normalization layers, and ReLU activation functions.

[0059] According to an embodiment of this disclosure, in step S103, the attention weights in the first feature map are adjusted based on the cross-channel attention strategy to obtain the adjusted attention weights, and the first feature map with the adjusted attention weights is used as the second feature map.

[0060] As an example, step S103 may further include: adjusting the attention weight of each channel according to formulas (3) to (5) which will be described later for each feature included in the deep and shallow features in the first feature map, to obtain the adjusted attention weight.

[0061] In other words, channel interaction attention redistribution can be achieved through step S103. Specifically, based on the extracted feature map containing image features, a channel interaction attention mechanism is used to adaptively adjust the channel weights in the feature map. For example, by using, but not limited to, one-dimensional (1D) convolution to model the information interaction between adjacent channels, the feature attention relationship of key regions is captured and the weight information is redistributed.

[0062] For example, a specific example of step S103 can be as follows:

[0063] First, a fast 1D convolutional network is used to extract semantically rich deep and shallow features. Then, the proposed Cross-channel Interaction Strategy (CIS) is employed to calculate the attention weights for each channel of both global and local features, thereby achieving long-range cross-channel interaction within a deep feature layer with high channel dimensionality. This is based on the interaction coverage (the kernel of the one-dimensional convolution, such as...). Figure 2(as shown in the kernel) (In addition, Figure 5 It also illustrates the nonlinear mapping relationship between the kernel size (k) and the channel dimension C, extending the linear function φ(k) = γ*k + b to the nonlinear function C = φ(k) = 2. (γ*k+b) The formulas for calculating the number of nearest neighbor channels k and the original weight g are as follows:

[0064]

[0065] g=C1D {k} (g(χ)) (4)

[0066] Among them, |t| odd This represents the odd number closest to t, where γ and b are preset parameter values ​​(which can be, for example, but not limited to, 2 and 1 respectively). The tensor data represents the image data, C1D represents fast one-dimensional convolution, and g(χ) represents global average pooling (GAP) of the channel dimension of the image data. Figure 2 (as shown) and the expression for g(χ) is the following equation (5):

[0067]

[0068] Where i is a natural number ranging from 1 to the value of W, and j is a natural number ranging from 1 to the value of H.

[0069] Then, the calculated attention weights are normalized using the Sigmoid function, as shown in equation (6) below:

[0070] ω=σ(C1D {k} (g(χ))) (6)

[0071] Where σ(·) represents the Sigmoid nonlinear activation function.

[0072] Subsequently, the normalized attention weights are multiplied by the original feature map to complete the redistribution of channel weights. The expression of the adjusted feature map is as follows (7):

[0073] F′=F·σ(g) (7)

[0074] According to embodiments of this disclosure, by modeling the information interaction between adjacent channels and applying a cross-channel interaction strategy to calculate the attention weight of each channel, the information in the feature map is redistributed, enhancing the model's focus on key features. In other words, by introducing a channel interaction attention mechanism, the channel weights in the feature map are adaptively adjusted, enhancing the model's ability to identify key features.

[0075] According to an embodiment of this disclosure, in step S104, the second feature map is input into a domain classifier network for image-level feature alignment (whose loss function can be expressed as, for example...). Figure 5 The domain classification loss shown) and the image-level multi-label classifier network for aligning features of a specific class (whose loss function can be expressed as, for example) Figure 5 The image-level classification loss shown is used to obtain the third feature map after feature alignment.

[0076] In other words, adversarial learning and image-level feature alignment can be achieved through step S104. Specifically, based on the redistribution of attention weight information, adversarial learning is used to further align the image-level feature distributions of the source and target domains globally and locally, while a multi-label classifier is used to align category-specific features.

[0077] As an example, step S104 may further include steps S1041 and S1042:

[0078] In step S1041, a domain label (e.g., ...) is added to each feature in the second feature map. Figure 5 The domain label (Y) shown indicates the correspondence between each feature and the source and target domains.

[0079] For example, each extracted feature is labeled with a neighborhood label. Specifically, one-hot encoding labels are used to represent the source domain of the input image, and the label assignment formula is shown in equation (8) below:

[0080]

[0081] In step S1042, after performing the following processes S41 and S42 in parallel, a third feature map is obtained by combining all features obtained after performing image-level feature alignment on both deep and shallow features in the second feature map and category-specific feature alignment on the deep features in the second feature map:

[0082] Regarding the processing S41, it includes: inputting the second feature map into a global domain classifier for deep features and a local domain classifier for shallow features, training the global domain classifier and the local domain classifier through an adversarial learning method, and adjusting the training parameters through a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data.

[0083] For example, firstly, design domain classifiers for global and local features respectively (e.g., Figure 5The two domain classifiers shown distinguish features between the source and target domains. For the global feature domain classifier, it can consist, for example, three sets of convolutional layers (Conv2D), batch normalization (BatchNorm2D), a non-linear activation function ReLU, and a Dropout regularization layer. Then, global average pooling is used to pool the spatial dimension of the feature map, reducing the feature dimensionality. The pooled features are flattened and resized, and the final domain classification result is output through a fully connected layer. For the local feature domain classifier, the domain classification result can be calculated using three convolutional layers and a sigmoid activation function, resulting in a simpler network structure and fewer computational parameters.

[0084] Then, the input feature X is processed according to the forward propagation process of the domain classifier constructed above, and the classification loss of the classification prediction result and the one-hot label tensor constructed above is calculated. Here, the loss function used to train the parameters of the global feature classifier is, for example, the Sigmoid focus loss function, to handle the imbalanced dataset and improve the classification performance of the model for the minority classes; the loss function used to train the parameters of the local feature domain classifier is, for example, the mean squared error loss (MSE), and the specific calculation formula is shown in the following equations (9) and (10):

[0085]

[0086] Among them, y i It is the actual value. The predicted value is logits, where N is the number of samples and α is the value. t It is a balancing parameter; γ is the key parameter that adjusts the weights of easy-to-classify and hard-to-classify samples; p i This represents the probability that the probability model, after Sigmoid transformation, predicts a positive class. Furthermore, through methods such as... Figure 5 The two gradient reversal layers (GRL) shown update the feature extraction network to learn domain-invariant features.

[0087] Regarding processing S42, it includes: inputting deep features included in the first feature map into an image-level multi-label classifier network, training the image-level multi-label classifier network through a supervised learning method, and adjusting the training parameters through an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0088] For example, image-level multi-label classifiers (e.g., Figure 5The construction process of the image-level classifier in AIAM is as follows: the deep features after attention redistribution are used as input, and the classification scores of each category are obtained through convolutional layers and global average pooling. Based on the scores and one-hot type image-level annotations, the binary cross-entropy loss is calculated. Under supervised training, category-specific features are learned to achieve category feature alignment between different domains. The specific calculation formula is shown in the following formula (11):

[0089]

[0090] Where σ(·) represents the Sigmoid function, y i It is the actual value. It is the predicted value logits.

[0091] According to embodiments of this disclosure, adversarial learning strategies and domain classifiers are used to further align image-level features of the source and target domains. The model is optimized through focus loss and MSE loss to improve its generalization ability to data from different domains, thereby enhancing the overall generalization ability of the model.

[0092] In addition, as an example, refer to Figure 5 The module used to implement the CIS strategy can be combined with the various classifiers in step S104 to form an attention-based image-level alignment module AIAM.

[0093] According to an embodiment of this disclosure, in step S105, the third feature map is input into a first prediction network for source domain instance-level feature alignment (whose loss function can be expressed as, for example...). Figure 5 The source domain RPN loss and detector head loss are shown, and a second prediction network is used for instance-level multi-label prediction in the target domain (whose loss function can be expressed as, for example...). Figure 5 The instance-level classification loss shown is used to obtain the fourth feature map.

[0094] As an example, step S105 may further include, after performing the following processes S51 and S52 in parallel, combining all features obtained after performing instance-level feature alignment on features associated with source domain data in the third feature map and instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map, to obtain a fourth feature map:

[0095] Regarding the processing S51, it includes: inputting the third feature map and source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of the source domain data.

[0096] For example, process S51 may further include steps S511 to S513:

[0097] In step S511, the third feature map, the first image-level annotation information, and the instance-level annotation information are input into the first prediction network to obtain the first prediction information corresponding to the first image-level annotation information and the second prediction information corresponding to the instance-level annotation information.

[0098] In step S512, a first loss function is calculated based on the first image-level annotation information, the instance-level annotation information, the first prediction information, and the second prediction information.

[0099] In step S513, the network parameters of the first prediction network are adjusted according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0100] In other words, source domain instance-level prediction alignment can be achieved by processing S51. Specifically, supervised learning (strongly supervised learning) is performed using annotation information (e.g., instance-level annotation information) in the source domain. Candidate regions are generated through a Region Proposal Network (RPN), and the classification loss and bounding box regression loss generated by the source domain image in the RPN and the detector head network are calculated. For example, the SoftMax cross-entropy loss function and the Smooth L1 loss function are used to optimize the model parameters to achieve instance-level feature alignment, thereby ensuring that the model has good predictive performance on the source domain data.

[0101] For example, a specific example of processing S51 can be as follows:

[0102] First, candidate regions are generated using RPN, and the binary cross-entropy classification loss and Smooth L1 regression loss of the RPN candidate boxes are calculated.

[0103] Then, the detection head is used to perform class prediction and bounding box regression on the candidate regions. The classification loss and regression loss are minimized using SoftMax cross-entropy and Smooth L1 respectively, and the network parameters are updated by backpropagation. The calculation formulas for the two loss functions are shown in equations (12) and (13) below:

[0104]

[0105]

[0106] Where s represents all source domain training data, y i Indicates the true category label, Let pre represent the predicted label, λ be the L2 regularization coefficient, θ represent the trainable parameters, and pre = (pre... x ,pre y ,pre w ,pre h ) represents the predicted bounding box coordinates, gt = (gt x gt y gtw gt h ) represents the coordinates of the actual bounding box.

[0107] According to the above embodiments of this disclosure, in the source domain, the RPN and the classification loss and regression loss generated by the detection head are calculated using existing annotation information to ensure the prediction accuracy of the model in the source domain.

[0108] Next, regarding processing S52, it includes: inputting the third feature map and target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of target domain data based on image-level category prediction.

[0109] For example, process S52 may further include steps S521 to S525:

[0110] In step S521, the deep features in the third feature map and the second image-level annotation information are input into the second prediction network to obtain the third prediction information and the fourth prediction information. The third prediction information represents the confidence score of the candidate box associated with the deep features containing the target object, and the fourth prediction information represents the prediction score of the target object in the candidate box belonging to a specific category.

[0111] In step S522, based on the third and fourth prediction information, the prediction score for the instance-specific category is reconstructed.

[0112] In step S523, based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information, the final prediction information based on image-level category prediction is obtained.

[0113] In step S524, a second loss function is calculated based on the second image-level annotation information and the final prediction information.

[0114] In step S525, the network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0115] In other words, by processing S52, it is possible to align the target domain instance-level reconstruction with image-level prediction. Specifically, weakly supervised learning is performed on the target domain data. The instance-level multi-label prediction module aggregates the candidate box scores of the RPN and the classification prediction results of the detection head to reconstruct the classification prediction matrix of instance-level features, while using binary cross-entropy loss for optimization.

[0116] Furthermore, on the target domain data, classification predictions are generated using instance-level features generated by the RPN and the detection head, and the candidate target scores from the RPN are used. With the classification prediction matrix x∈ from the detection head (in, It aggregates the total number of categories to achieve weakly supervised learning.

[0117] here, This represents the predicted probability score that the candidate boxes generated by the RPN contain the target object. This represents the probability score of the target object in the candidate box predicted by the detection head to belong to each category, and the two scores are compared with those of a multi-head classifier (e.g., Figure 5 The image-level classifier in the I2ITM bounding box constitutes the instance-level multi-label prediction module (i.e., Figure 5 The weighted adjusted category-specific prediction score is obtained from the I2ITM (in the I2ITM).

[0118] Specifically, for example, a specific example of processing S52 (the specific process is as follows) Figure 4 As shown, it will not be elaborated further here. Figure 4 The specific content can be as follows:

[0119] First, calculate the new candidate matrix based on the classification prediction matrix x of the detection head.

[0120] Here, according to x n confidence score o n Assign a specific target category along the row: if x n If the category index of the maximum value is i, then the score will be o. n Assign a value to the i-th class; if x n If the category index of the minimum value is j, then the score will be o. n The negative of the value is assigned to the j-th class; the predicted scores for all other classes are assigned to 0; finally, the target score matrix for the specific class is obtained. In this matrix, categories with high classification probabilities are assigned high confidence, categories with low probabilities are assigned low confidence, and all other categories are assigned zero confidence, thus obtaining a category probability distribution matrix that is highly indicative of category information.

[0121] Then, the SoftMax function is used to obtain new category-specific prediction matrices by weighting the initial score matrix x. Normalization is performed along both rows and columns, and a weighted suggestion score is calculated. Finally, the SoftMax(x) and... Element-wise multiplication is performed between them to obtain the weighted and adjusted prediction matrix. The summation of these sums yields the final image-level prediction result P. Formally, the image-level prediction aggregation can be expressed as equation (14):

[0122]

[0123] in, It is the predicted probability of category c (indicating whether the current image contains a target of category c), σ row (·) and σ col (·) represents the SoftMax operation along the row and column respectively, and (·) represents the element-wise multiplication operation.

[0124] Finally, under the premise of image-level multi-label prediction and one-hot label encoding, the model is trained using binary cross-entropy loss and optimized by backpropagation. Weakly supervised training is used to reduce the domain differences of instance-level features.

[0125] According to the above embodiments of this disclosure, weakly supervised learning is achieved in the target domain by combining the prediction of RPN and detection head through instance-level multi-label prediction module, and the impact of domain differences is effectively reduced by optimizing binary cross-entropy loss.

[0126] According to an embodiment of this disclosure, in step S106, the model parameters of the cross-domain object detection model are adjusted based on the source domain data, the target domain data, and the first to fourth feature maps to obtain the trained cross-domain object detection model.

[0127] As an example, based on the source domain data, the target domain data, and the first to fourth feature maps, we can obtain, as follows: Figure 5 The five losses shown are used to adjust the parameters of the entire model by minimizing the sum L of these five losses. For example, The expression for can be shown in equation (15) below:

[0128]

[0129] Wherein, the loss weight λ dc , λ ic and λ cls These are preset parameters, which can be set to 1, 0.1, and 1 respectively.

[0130] According to embodiments of this disclosure, weakly supervised cross-domain object detection that integrates adversarial learning and self-aware hierarchical alignment is achieved through the training method described above. By utilizing the object detection techniques for different cross-domain scenarios disclosed in this disclosure, the problem of significant performance degradation of the model caused by domain differences is solved.

[0131] Furthermore, by utilizing theoretical models such as deep learning methods, adversarial learning methods, and attention mechanisms, and integrating cross-domain, adversarial feature learning, and holistic to hierarchical alignment, an end-to-end cross-domain object detection framework is constructed. This improves the accuracy of object detection in different cross-domain scenarios and can further serve various application scenarios such as traffic scenarios and virtual reality scenarios.

[0132] Here, it should be noted that the above regarding Figure 1 The descriptions of the various steps are merely exemplary, and the steps in the method according to this disclosure are not limited thereto.

[0133] Furthermore, this disclosure provides a cross-domain object detection method for image data. This cross-domain object detection method includes: acquiring image data to be detected; inputting the image data to be detected into a cross-domain object detection model for images to obtain corresponding image detection data. Here, the cross-domain object detection model is trained using the training method described above.

[0134] Next, refer to Figure 6 This describes a training apparatus for a cross-domain object detection model for images according to embodiments of the present disclosure.

[0135] Figure 6 This is a block diagram illustrating a training apparatus 600 for a cross-domain object detection model for images according to an embodiment of the present disclosure. For example, the cross-domain object detection model includes a deep learning network, a domain classifier network, an image-level multi-label classifier network, a first prediction network, and a second prediction network.

[0136] Reference Figure 6 The training apparatus 600 for a cross-domain object detection model for images according to an embodiment of the present disclosure may include a data acquisition module 610, a feature extraction module 620, a weight adjustment module 630, a first feature alignment module 640, a second feature alignment module 650, and a parameter adjustment module 660.

[0137] According to embodiments of this disclosure, the data acquisition module 610 can perform the following operations: acquire source domain data and target domain data. Here, the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data.

[0138] Furthermore, as an example, the source domain image annotation data may include first image-level annotation information and instance-level annotation information, and the target domain image annotation data may include second image-level annotation information. That is, the target domain image annotation data does not include instance-level annotation information, but this disclosure is not limited thereto, and may also include a small amount of instance-level annotation information that is insufficient for performing supervised learning methods.

[0139] According to an embodiment of this disclosure, the feature extraction module 620 can perform the following: input source domain data and target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image.

[0140] According to an embodiment of this disclosure, the weight adjustment module 630 can perform the following: based on a cross-channel attention strategy, adjust the attention weights in the first feature map to obtain the adjusted attention weights, and use the first feature map with the adjusted attention weights as the second feature map.

[0141] As an example, the above operation performed by the weight adjustment module 630 may further include: adjusting the attention weight of each channel according to the above formulas (3) to (5) for each feature included in the deep features and shallow features in the first feature map, so as to obtain the adjusted attention weight.

[0142] According to an embodiment of this disclosure, the first feature alignment module 640 can perform the following: inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment.

[0143] As an example, the above-described operations performed by the first feature alignment module 640 may further include the following operations 641) and 642):

[0144] In operation 641), a domain label is added to each feature in the second feature map. The domain label indicates the correspondence between each feature and the source domain and the target domain.

[0145] In operation 642), after performing the following processes (1) and (2) in parallel, a third feature map is obtained by combining all features after performing image-level feature alignment on both deep and shallow features in the second feature map and class-specific feature alignment on the deep features in the second feature map:

[0146] (1) Input the second feature map into the global domain classifier for deep features and the local domain classifier for shallow features. Train the global domain classifier and the local domain classifier through adversarial learning and adjust the training parameters through the domain classification loss function to achieve image-level feature alignment between source domain data and target domain data.

[0147] (2) Input the deep features included in the first feature map into the image-level multi-label classifier network, train the image-level multi-label classifier network through supervised learning method and adjust the training parameters through the image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0148] According to an embodiment of this disclosure, the second feature alignment module 650 can perform the following: inputting a third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map.

[0149] As an example, the operations performed by the second feature alignment module 650 may further include: after performing the following processes 651) and 652) in parallel, combining all features obtained after performing instance-level feature alignment on features associated with source domain data in the third feature map and instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map, to obtain a fourth feature map:

[0150] For processing 651), the third feature map and the source domain data are input into the first prediction network for source domain instance-level feature alignment. The first prediction network is trained by a supervised learning method to achieve instance-level feature alignment of the source domain data.

[0151] For example, the above process 651) may further include operations 6511) and 6512):

[0152] In operation 6511), the third feature map, the first image-level annotation information, and the instance-level annotation information are input into the first prediction network to obtain the first prediction information corresponding to the first image-level annotation information and the second prediction information corresponding to the instance-level annotation information.

[0153] In operation 6512), based on the first image-level annotation information, instance-level annotation information, first prediction information, and second prediction information, a first loss function is calculated, and the network parameters of the first prediction network are adjusted according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0154] For processing 652), the third feature map and the target domain data are input into the second prediction network for instance-level category prediction of the target domain. The second prediction network is trained by a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0155] For example, the above process 652) may further include operations 6521) and 6524):

[0156] In operation 6521), the deep features in the third feature map and the second image-level annotation information are input into the second prediction network to obtain the third prediction information and the fourth prediction information.

[0157] Here, the third prediction information represents the confidence score of the target object contained in the candidate box associated with the deep features, and the fourth prediction information represents the prediction score of the target object contained in the candidate box belonging to a specific category.

[0158] In operation 6522), based on the third and fourth prediction information, the prediction score for the specific category at the instance level is reconstructed.

[0159] In operation 6523), based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information, the final prediction information based on image-level category prediction is obtained.

[0160] In operation 6524), based on the second image-level annotation information and the final prediction information, a second loss function is calculated, and the network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0161] According to an embodiment of this disclosure, the parameter adjustment module 660 can perform the following: adjust the model parameters of the cross-domain object detection model based on the source domain data, the target domain data, and the first to fourth feature maps, to obtain the trained cross-domain object detection model.

[0162] It should be noted that the operations performed on the above structural frames can be compared with those in the reference section. Figure 1 The related content is similar, so I will not repeat it here.

[0163] Figure 7 This is a block diagram illustrating a computing device 700 according to an embodiment of the present disclosure.

[0164] Reference Figure 7 The computing device 700 according to embodiments of the present disclosure may include a processor 710 and a memory 720. The processor 710 may include (but is not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 720 may store computer-executable instructions to be executed by the processor 710. The memory 720 includes high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 710 executes the computer-executable instructions stored in the memory 720, the training method and apparatus for a cross-domain object detection model for images and the cross-domain object detection method described above can be implemented.

[0165] The training method and apparatus for a cross-domain object detection model for images, and the cross-domain object detection method according to embodiments of this disclosure, can be written as a computer program / instructions to form a computer program product and stored on a computer-readable storage medium. When the computer program / instructions are executed by a processor, the training method and apparatus for a cross-domain object detection model for images and the cross-domain object detection method described above can be implemented. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, the electronic device / server is enabled to perform the training method and apparatus for a cross-domain object detection model for images and the cross-domain object detection method described above. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0166] The training method and apparatus for cross-domain object detection models for images according to embodiments of the present disclosure, and the cross-domain object detection method, realize a cross-domain weakly supervised object detection algorithm for image data by constructing a cross-domain strategy that integrates adversarial learning and self-aware hierarchical alignment, thereby improving the accuracy of image detection tasks in cross-domain scenarios.

[0167] Furthermore, the training method and apparatus for cross-domain object detection models for images and the cross-domain object detection method according to embodiments of this disclosure solve the problem of a significant decrease in model detection performance caused by domain differences.

[0168] Furthermore, since features with specific category information and high discriminativeness usually play a key role in predicting output for finer-grained target detection tasks, while suppressing model overfitting caused by unrelated regions, the method described above can more effectively complete such finer-grained target detection tasks.

[0169] While some embodiments of this disclosure have been disclosed and described, those skilled in the art will understand that modifications and variations may be made to these embodiments without departing from the concept and spirit of this disclosure, which is defined by the claims and their equivalents.

Claims

1. A training method for a cross-domain object detection model for images, characterized in that, The training method includes: Acquire source domain data and target domain data, wherein the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data; The source domain data and the target domain data are input into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image. Based on the cross-channel attention strategy, the attention weights in the first feature map are adjusted to obtain the adjusted attention weights, and the first feature map with the adjusted attention weights is used as the second feature map. The second feature map is input into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment. The third feature map is input into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map. Based on the source domain data, the target domain data, and the first to fourth feature maps, the model parameters of the cross-domain object detection model are adjusted to obtain the trained cross-domain object detection model. The cross-domain object detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network, and the second prediction network. The step of adjusting the attention weights in the first feature map based on the cross-channel cross-attention strategy to obtain the adjusted attention weights includes: For each feature included in the deep and shallow features of the first feature map, the attention weight of each channel is adjusted according to the following formula to obtain the adjusted attention weight: ， in, Indicates attention weights, Represents fast one-dimensional convolution. Indicates the number of adjacent channels and The expression is: , Represents the global average pooling of the channel dimension of the image data and The expression is: , in, Indicates channel dimension, and All are preset parameter values. Represents the nearest odd number. Tensor data representing image data and These represent the width and height of the tensor, respectively. i The value range is from 1 to 1. The value of a natural number, j The value range is 1 to The value of a natural number, The step of inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map includes: After performing the following processing in parallel, the fourth feature map is obtained by combining all features obtained after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map: The third feature map and the source domain data are input into a first prediction network for source domain instance-level feature alignment. The first prediction network is trained using a supervised learning method to achieve instance-level feature alignment of the source domain data. The third feature map and the target domain data are input into a second prediction network for instance-level category prediction in the target domain. The second prediction network is trained using a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

2. The training method according to claim 1, characterized in that, The step of inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain the feature-aligned third feature map includes: Add a domain label to each feature in the second feature map, the domain label indicating the correspondence between each feature and the source domain and the target domain; After performing the following processing in parallel, the third feature map is obtained by combining all features resulting from image-level feature alignment performed on both deep and shallow features in the second feature map and category-specific feature alignment performed on the deep features in the second feature map: The second feature map is input into a global domain classifier for deep features and a local domain classifier for shallow features. Adversarial learning is used to train both the global and local domain classifiers, and the training parameters are adjusted using a domain classification loss function to achieve image-level feature alignment between the source and target domain data. The deep features included in the first feature map are input into the image-level multi-label classifier network. The image-level multi-label classifier network is trained by a supervised learning method, and the training parameters are adjusted by an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

3. The training method according to claim 1, characterized in that, The source domain image annotation data includes first image-level annotation information and instance-level annotation information. The step of inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network using a supervised learning method to achieve instance-level feature alignment of the source domain data, includes: The third feature map, the first image-level annotation information, and the instance-level annotation information are input into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information. Based on the first image-level annotation information, the instance-level annotation information, the first prediction information, and the second prediction information, a first loss function is calculated, and the network parameters of the first prediction network are adjusted according to the first loss function to achieve instance-level feature alignment of the source domain data.

4. The training method according to claim 1, characterized in that, The target domain image annotation data includes second image-level annotation information, and the step of inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network using a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction includes: The deep features in the third feature map and the second image-level annotation information are input into the second prediction network to obtain third prediction information and fourth prediction information. The third prediction information represents the confidence score of the candidate box associated with the deep features containing the target object, and the fourth prediction information represents the prediction score of the target object in the candidate box belonging to a specific category. Based on the third and fourth prediction information, the prediction score for a specific category at the instance level is reconstructed. Based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information, the final prediction information based on image-level category prediction is obtained. Based on the second image-level annotation information and the final prediction information, a second loss function is calculated, and the network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

5. A training device for a cross-domain object detection model for images, characterized in that, The training device includes: The data acquisition module is configured to acquire source domain data and target domain data, wherein the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data. The feature extraction module is configured to: input the source domain data and the target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; The weight adjustment module is configured to: adjust the attention weights in the first feature map based on a cross-channel attention strategy to obtain the adjusted attention weights, and use the first feature map with the adjusted attention weights as the second feature map. The first feature alignment module is configured to input the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment, to obtain a third feature map after feature alignment. The second feature alignment module is configured to input the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map. The parameter adjustment module is configured to: adjust the model parameters of the cross-domain object detection model based on the source domain data, the target domain data, and the first to fourth feature maps, to obtain the trained cross-domain object detection model. The cross-domain object detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network, and the second prediction network. The weight adjustment module adjusts the attention weights in the first feature map based on a cross-channel attention strategy to obtain the adjusted attention weights. The operation includes: For each feature included in the deep and shallow features of the first feature map, the attention weight of each channel is adjusted according to the following formula to obtain the adjusted attention weight: ， in, Indicates attention weights, Represents fast one-dimensional convolution. Indicates the number of adjacent channels and The expression is: , Represents the global average pooling of the channel dimension of the image data and The expression is: , in, Indicates channel dimension, and All are preset parameter values. Represents the nearest odd number. Tensor data representing image data and These represent the width and height of the tensor, respectively. i The value range is from 1 to 1. The value of a natural number, j The value range is 1 to The value of a natural number, The second feature alignment module inputs the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain the fourth feature map. The operation includes: After performing the following processing in parallel, the fourth feature map is obtained by combining all features obtained after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map: The third feature map and the source domain data are input into a first prediction network for source domain instance-level feature alignment. The first prediction network is trained using a supervised learning method to achieve instance-level feature alignment of the source domain data. The third feature map and the target domain data are input into a second prediction network for instance-level category prediction in the target domain. The second prediction network is trained using a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

6. A method for cross-domain target detection of image data, characterized in that, The cross-domain target detection method includes: Acquire the image data to be detected; The image data to be detected is input into a cross-domain object detection model for images to obtain the corresponding image detection data. The cross-domain target detection model is trained using the training method described in any one of claims 1 to 4.

7. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the training method for a cross-domain object detection model for images as described in any one of claims 1 to 4, and the cross-domain object detection method for image data as described in claim 6.

8. A computing device, characterized in that, The computing device includes: at least one processor; at least one memory storing computer-executable instructions, wherein, when executed by the at least one processor, the computer-executable instructions cause the at least one processor to perform the training method for a cross-domain object detection model for an image as claimed in any one of claims 1 to 4 and the cross-domain object detection method for image data as claimed in claim 6.

Images