Training method and device of cross-domain target detection model for image and cross-domain target detection method

Abstract

The invention discloses a training method and device of a cross-domain target detection model for an image and a cross-domain target detection method. The training method comprises the following steps: acquiring source domain data and target domain data; inputting the source domain data and the target domain data into a deep learning network to obtain a first feature map; based on a cross-channel cross attention strategy, adjusting the attention weight in the first feature map to obtain a second feature map with the adjusted attention weight; inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map; inputting the third feature map into a first prediction network used for source domain instance-level feature alignment and a second prediction network used for target domain instance-level multi-label prediction to obtain a fourth feature map; and according to the source domain data, the target domain data and the first to fourth feature maps, adjusting model parameters to obtain a trained cross-domain target detection model.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computer vision technology, and more specifically, to a training method and device for a cross-domain object detection model for an image and a cross-domain object detection method. Background Art

[0002] In related fields, with the rapid development of computer vision technology, object detection has become an important research direction in the field of computer vision. The purpose of object detection is to automatically identify objects in images or videos through some means, and determine their locations and categories. Object detection technology has a wide range of applications, such as intelligent transportation systems, video surveillance, autonomous driving, etc.

[0003] In order to achieve target detection, commonly used means can be image processing and machine learning. With the development of machine learning technology, the accuracy and efficiency of target detection are constantly improving, but it also faces a series of problems. For example, as a training data set for training models, sufficient annotated data sets are usually required to enable the training results to achieve high accuracy. However, manual labeling requires a lot of manpower and material resources, and due to the influence of environmental conditions such as lighting and climatic conditions, the data in actual application scenarios is more complex and changeable than the collected data sets. When migrating models in different scenarios, the model performance will be greatly reduced due to the large differences in field distribution. Summary of the invention

[0004] The embodiments of the present disclosure provide a training method and device for a cross-domain target detection model for images, and a cross-domain target detection method. By constructing a cross-domain strategy that integrates adversarial learning and self-perceived hierarchical alignment, a cross-domain weakly supervised target detection algorithm for image data is implemented, thereby improving the accuracy of image detection tasks in cross-domain scenarios.

[0005] In one general aspect, a training method for a cross-domain object detection model for an image is provided, the training method comprising: obtaining source domain data and target domain data, the source domain data comprising source domain image data and source domain image annotation data, the target domain data comprising target domain image data and target domain image annotation data; inputting the source domain data and the target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; adjusting the attention weight in the first feature map based on a cross-channel cross-attention strategy to obtain an adjusted attention weight, and using the first feature map with the adjusted attention weight as a second feature map; inputting the second feature map into a pre-trained deep learning network for A domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment are used to obtain a third feature map after feature alignment; the third feature map is input into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; according to the source domain data and the target domain data and the first feature map to the fourth feature map, the model parameters of the cross-domain target detection model are adjusted to obtain a trained cross-domain target detection model, wherein the cross-domain target detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network and the second prediction network.

[0006] Optionally, the step of adjusting the attention weight in the first feature map based on the cross-channel cross attention strategy to obtain the adjusted attention weight may include: for each feature in the deep features and shallow features included in the first feature map, adjusting the attention weight of each channel according to the following formula to obtain the adjusted attention weight:

[0007] g=C1D {k} (g(χ))

[0008] Among them, g represents the attention weight, C1D represents fast one-dimensional convolution, k represents the number of adjacent channels and the expression of k is: g(χ) represents the global average pooling of the channel dimension of the image data and the expression of g(χ) is: Among them, C represents the channel dimension, γ and b are preset parameter values, | | odd represents the nearest odd number, The tensor data representing the image data, W and H represent the width and height of the tensor respectively, i is a natural number ranging from 1 to W, and j is a natural number ranging from 1 to H.

[0009] Optionally, the step of inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment may include: adding a domain label to each feature in the second feature map, the domain label indicating the correspondence between each feature and the source domain and the target domain; after performing the following processing in parallel, the third feature map is obtained by combining all features after performing image-level feature alignment on both deep features and shallow features in the second feature map and performing specific category feature alignment on the deep features in the second feature map. Feature map: input the second feature map to a global domain classifier for deep features and a local domain classifier for shallow features, train the global domain classifier and the local domain classifier through an adversarial learning method, and adjust the training parameters through a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data; and input the deep features included in the first feature map to the image-level multi-label classifier network, train the image-level multi-label classifier network through a supervised learning method, and adjust the training parameters through an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0010] Optionally, the step of inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map may include: after performing the following processing in parallel, obtaining the fourth feature map by combining all features after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map: inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of source domain data; inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of target domain data based on image-level category prediction.

[0011] Optionally, the source domain image annotation data may include first image-level annotation information and instance-level annotation information, and the step of inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of the source domain data may include: inputting the third feature map, the first image-level annotation information, and the instance-level annotation information into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information; calculating a first loss function based on the first image-level annotation information and the instance-level annotation information as well as the first prediction information and the second prediction information, and adjusting the network parameters of the first prediction network according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0012] Optionally, the target domain image annotation data may include second image-level annotation information, and the step of inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction may include: inputting the deep features in the third feature map and the second image-level annotation information into the second prediction network to obtain third prediction information and fourth prediction information, the third prediction information representing a confidence score of the target object contained in the candidate box associated with the deep features, and the fourth prediction information representing a prediction score that the target object contained in the candidate box belongs to a specific category; reconstructing an instance-level specific category prediction score based on the third prediction information and the fourth prediction information; obtaining final prediction information based on image-level category prediction based on the instance-level specific category prediction score, the third prediction information and the fourth prediction information; calculating a second loss function based on the second image-level annotation information and the final prediction information, and adjusting the network parameters of the second prediction network according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0013] In another general aspect, a training device for a cross-domain object detection model for images is provided, the training device comprising: a data acquisition module, configured to: acquire source domain data and target domain data, the source domain data comprising source domain image data and source domain image annotation data, the target domain data comprising target domain image data and target domain image annotation data; a feature extraction module, configured to: input the source domain data and target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; a weight adjustment module, configured to: adjust the attention weight in the first feature map based on a cross-channel cross-attention strategy to obtain an adjusted attention weight, and use the first feature map with the adjusted attention weight as a second feature map; a first feature alignment module, configured to: The second feature map is input into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment; the second feature alignment module is configured to: input the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; the parameter adjustment module is configured to: adjust the model parameters of the cross-domain target detection model according to the source domain data and the target domain data and the first feature map to the fourth feature map to obtain a trained cross-domain target detection model, wherein the cross-domain target detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network and the second prediction network.

[0014] Optionally, the weight adjustment module adjusts the attention weight in the first feature map based on a cross-channel cross attention strategy, and the operation of obtaining the adjusted attention weight may include: for each feature in the deep features and shallow features included in the first feature map, adjusting the attention weight of each channel according to the following formula to obtain the adjusted attention weight:

[0015] g=C1D {k} (g(χ))

[0016] Among them, g represents the attention weight, C1D represents fast one-dimensional convolution, k represents the number of adjacent channels and the expression of k is: g(χ) represents the global average pooling of the channel dimension of the image data and the expression of g(χ) is: Among them, C represents the channel dimension, γ and b are preset parameter values, || odd represents the nearest odd number, The tensor data representing the image data, W and H represent the width and height of the tensor respectively, i is a natural number ranging from 1 to W, and j is a natural number ranging from 1 to H.

[0017] Optionally, the first feature alignment module inputs the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment, and the operation of obtaining the third feature map after feature alignment may include: adding a domain label to each feature in the second feature map, the domain label indicating the correspondence between each feature and the source domain and the target domain; after performing the following processing in parallel, by combining all features after performing image-level feature alignment on both deep features and shallow features in the second feature map and performing specific category feature alignment on the deep features in the second feature map, the third feature map after feature alignment is obtained. The third feature map: input the second feature map to a global domain classifier for deep features and a local domain classifier for shallow features, train the global domain classifier and the local domain classifier by an adversarial learning method, and adjust the training parameters by a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data; and input the deep features included in the first feature map to the image-level multi-label classifier network, train the image-level multi-label classifier network by a supervised learning method, and adjust the training parameters by an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0018] Optionally, the second feature alignment module inputs the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction, and the operation of obtaining the fourth feature map may include: after performing the following processing in parallel, the fourth feature map is obtained by combining all features after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map: inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of source domain data; inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of target domain data based on image-level category prediction.

[0019] Optionally, the source domain image annotation data may include first image-level annotation information and instance-level annotation information, and the second feature alignment module inputs the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and trains the first prediction network through a supervised learning method to achieve instance-level feature alignment of source domain data. The operation may include: inputting the third feature map, the first image-level annotation information, and the instance-level annotation information into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information; calculating a first loss function based on the first image-level annotation information and the instance-level annotation information as well as the first prediction information and the second prediction information, and adjusting the network parameters of the first prediction network according to the first loss function to achieve instance-level feature alignment of source domain data.

[0020] Optionally, the target domain image annotation data may include second image-level annotation information, and the second feature alignment module inputs the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and trains the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction. The operation may include: inputting the deep features in the third feature map and the second image-level annotation information into the second prediction network to obtain third prediction information and fourth prediction information, wherein the third prediction information represents a confidence score that the target object is contained in the candidate box associated with the deep features, and the fourth prediction information represents a prediction score that the target object contained in the candidate box belongs to a specific category; based on the third prediction information and the fourth prediction information, reconstructing an instance-level specific category prediction score; based on the instance-level specific category prediction score, the third prediction information and the fourth prediction information, obtaining final prediction information based on image-level category prediction; based on the second image-level annotation information and the final prediction information, calculating a second loss function, and adjusting the network parameters of the second prediction network according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0021] In another general aspect, a cross-domain target detection method for image data is provided, the cross-domain target detection method comprising: acquiring image data to be detected; inputting the image data to be detected into a cross-domain target detection model for the image to obtain corresponding image detection data, wherein the cross-domain target detection model is trained using the training method described above.

[0022] In another general aspect, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the training method for a cross-domain object detection model for an image and the cross-domain object detection method for image data as described above.

[0023] In another general aspect, a computer-readable storage medium is provided, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, enables the electronic device / server to execute the training method of the cross-domain target detection model for images and the cross-domain target detection method for image data as described above.

[0024] In another general aspect, a computing device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, cause the at least one processor to execute the training method for a cross-domain object detection model for images and the cross-domain object detection method for image data as described above.

[0025] According to the training method and device of the cross-domain target detection model for images and the cross-domain target detection method of the embodiments of the present disclosure, a cross-domain weakly supervised target detection algorithm for image data is implemented by constructing a cross-domain strategy that integrates adversarial learning and self-perception hierarchical alignment, thereby improving the accuracy of the detection task for images in cross-domain scenarios. In addition, according to the training method and device of the cross-domain target detection model for images and the cross-domain target detection method of the embodiments of the present disclosure, the problem of a significant decrease in model detection performance caused by domain differences is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects and features of the embodiments of the present disclosure will become more apparent through the following description in conjunction with the accompanying drawings showing the embodiments, in which:

[0027] Figure 1 is a flowchart illustrating a method for training a cross-domain object detection model for an image according to an embodiment of the present disclosure;

[0028] Figure 2 is an example structural diagram showing a channel interaction attention network according to the present disclosure;

[0029] Figure 3 is an example structural diagram showing an instance-level weakly supervised network according to an embodiment of the present disclosure;

[0030] Figure 4 is an example flow chart illustrating an instance-level weak supervision method according to an embodiment of the present disclosure;

[0031] Figure 5 is a schematic diagram showing a schematic network structure according to an embodiment of the present disclosure;

[0032] Figure 6 is a block diagram showing a training apparatus for a cross-domain object detection model for an image according to an embodiment of the present disclosure;

[0033] Figure 7 is a block diagram illustrating a computing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following specific embodiments are provided to help the reader obtain a comprehensive understanding of the methods, devices and / or systems described herein. However, after understanding the disclosure of the present application, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, for greater clarity and simplicity, the description of features known in the art may be omitted.

[0035] Reference will now be made in detail to the embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments will be described below with reference to the drawings in order to explain the present disclosure.

[0036] As mentioned above, in the related art, existing neural network-based object detection methods often require sufficient labeled datasets to achieve higher accuracy. However, the data in actual application scenarios is more complex and changeable than the collected datasets, and the performance will be greatly reduced when performing model migration due to the large differences in domain distribution. Therefore, weakly supervised domain adaptation object detection methods that use additional image-level annotations to transfer information learned on labeled datasets to new domains have received widespread attention in various fields.

[0037] The learning scenario based on weakly supervised domain adaptation limits the dataset with a large number of labeled samples as the source domain, and the dataset with no or only a small number of labeled samples as the target domain. The current main domain adaptation methods include the following two: domain adaptation classification based on adversarial learning and domain adaptation based on self-training.

[0038] Specifically, for the former, the typical network is the Domain Adaptation Neural Network (DANN) based on adversarial learning. For the latter, this type of method generates high-quality pseudo-labels on the target domain data to assist in training, so as to make full use of the effective information in the target domain. It is extremely dependent on the generation quality of pseudo-labels. Typical methods include the Unbiased Mean Teacher (UMT) method and the Adaptive Teacher (AT) method based on the teacher-student distillation model.

[0039] However, these methods may have a series of problems, such as the inability to accurately capture the complex distribution differences of local features, difficulty in aligning local domains, and poor model generalization.

[0040] The present invention discloses a training method and device for a cross-domain target detection model for images and a cross-domain target detection method. By adopting adversarial learning and self-perception hierarchical alignment strategies, a cross-domain (or cross-domain) weakly supervised target detection algorithm for images is implemented, which can improve the accuracy of target detection tasks in cross-domain scenarios and effectively solve the problem of a significant decrease in model detection performance caused by domain differences.

[0041] Refer to the following Figures 1 to 7 The present invention describes in detail a method and device for training a cross-domain object detection model for an image and a cross-domain object detection method according to an embodiment of the present invention.

[0042] First, refer to Figures 1 to 5 To describe a training method for a cross-domain object detection model for images according to an embodiment of the present disclosure.

[0043] Figure 1 is a flowchart illustrating a method 100 for training a cross-domain object detection model for an image according to an embodiment of the present disclosure. Figure 2 is an example structural diagram showing a channel interaction attention network according to the present disclosure. Figure 3 is an example structural diagram showing an instance-level weakly supervised network according to an embodiment of the present disclosure.

[0044] Figure 4 is an example flow chart illustrating an instance-level weak supervision method according to an embodiment of the present disclosure. Figure 5 is a schematic diagram showing a schematic network structure according to an embodiment of the present disclosure.

[0045] As an example, a cross-domain object detection model for an image includes a deep learning network, a domain classifier network, an image-level multi-label classifier network, a first prediction network, and a second prediction network.

[0046] Reference Figure 1According to an embodiment of the present disclosure, in step S101, source domain data and target domain data are acquired. Here, the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data.

[0047] In addition, the source domain image annotation data includes image-level annotation information (i.e., category annotation information) and instance-level annotation information (i.e., annotation information corresponding to bounding box regression). The target domain image annotation data includes image-level annotation information, but does not include or includes less instance-level annotation information. In addition, the image-level annotation information included in the source domain may also be referred to as first image-level annotation information, and the image-level annotation information included in the target domain may also be referred to as second image-level annotation information.

[0048] According to an embodiment of the present disclosure, in step S102, source domain data and target domain data are input into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image.

[0049] For example, the first feature map may include only a deep feature map, or may include a deep feature map and a shallow feature map. That is, as an example, the deep features may be output to the next network as the output of the deep learning network, and the shallow features may be extracted when the other network needs the features. As another example, the deep learning network may also output a feature map including both deep features (also called global features) and shallow features (also called local features).

[0050] That is to say, step S102 can realize image feature extraction of the source domain and the target domain.

[0051] Specifically, first, the source domain image data (hereinafter referred to as the source domain image) and the target domain image data (hereinafter referred to as the target domain image) as input are represented as numerical tensors; then, a pre-trained backbone network (for example, Figure 5 The schematic BackBone backbone network shown in the figure performs convolution operations on the image tensor to extract image features.

[0052] For example, a specific example of step S102 may be as follows:

[0053] First, the input source domain image and target domain image can be represented as numerical tensors [C, W, H], where C represents the three channels of the image pixels, and W and H represent the width and height of the tensor respectively. Select image tensors from the source domain S and the target domain T respectively and combine them to obtain the model input tensor representation The specific expression is shown in the following formula (1):

[0054] X=(Xt,Xs)=[2N,C,W,H] (1)

[0055] Among them, Xt and Xs represent multi-dimensional tensors from the target domain and source domain respectively, 2N represents the total batch size of the input images, the first N images are from the target domain, and the next N images are from the source domain.

[0056] Then, the input image tensor is subjected to the convolution layer, batch normalization layer and ReLU activation function of the pre-trained backbone network for feature extraction to obtain the corresponding feature representation F. The specific expression is shown in the following formula (2):

[0057] F = BackBone (Xt, Xs) (2)

[0058] The step of extracting image features by inputting image data into a deep convolutional neural network can extract high-level semantic features of the image through multiple stacked convolutional layers, batch normalization layers, and ReLU activation functions.

[0059] According to an embodiment of the present disclosure, in step S103, based on a cross-channel cross-attention strategy, the attention weight in the first feature map is adjusted to obtain an adjusted attention weight, and the first feature map with the adjusted attention weight is used as the second feature map.

[0060] As an example, step S103 may further include: for each feature of the deep features and shallow features included in the first feature map, adjusting the attention weight of each channel according to formulas (3) to (5) to be described later, to obtain an adjusted attention weight.

[0061] That is, channel interaction attention reallocation can be achieved through step S103. Specifically, based on the extracted feature map containing image features, the channel interaction attention mechanism is used to adaptively adjust the channel weights in the feature map. For example, by using, for example but not limited to, one-dimensional (1D) convolution to model the information interaction between adjacent channels, the feature attention relationship of the key area is captured and the weight information is reallocated.

[0062] For example, a specific example of step S103 may be as follows:

[0063] First, a fast 1D convolutional network is used to extract semantically rich deep and shallow features, and the proposed cross-channel interaction strategy (CIS) is used to calculate the attention weights of each channel of global and local features, so as to achieve long-range cross-channel interaction in the deep feature layer with high channel dimension. Figure 2) (In addition, Figure 5 It also shows the nonlinear mapping relationship between the kernel and padding) size) and the channel dimension C, expanding the linear function φ(k) = γ*k+b to a nonlinear function C = φ(k) = 2 (γ*k+b) , the calculation formula of the number of neighbor channels k and the original weight g is as follows:

[0064]

[0065] g=C1D {k} (g(χ)) (4)

[0066] Where |t| odd represents the odd number closest to t, γ and b are both preset parameter values ​​(which may be, for example but not limited to, set to 2 and 1 respectively), represents the tensor data of image data, C1D represents fast one-dimensional convolution, and g(χ) represents the global average pooling (GAP, such as Figure 2 The expression of g(χ) is as follows:

[0067]

[0068] Among them, i is a natural number with a value ranging from 1 to W, and j is a natural number with a value ranging from 1 to H.

[0069] Then, the calculated attention weight is normalized by the Sigmoid function, and the calculation formula is shown in the following formula (6):

[0070] ω=σ(C1D {k} (g(χ))) (6)

[0071] Among them, σ(·) represents the Sigmoid nonlinear activation function.

[0072] Subsequently, the normalized attention weight is multiplied by the original feature map to complete the redistribution of the channel weights. The expression of the adjusted feature map is as follows:

[0073] F′=F·σ(g) (7)

[0074] According to the embodiments of the present disclosure, by modeling the information interaction between adjacent channels and applying the cross-channel interaction strategy to calculate the attention weight of each channel, the redistribution of information in the feature map is achieved, and the model's attention to key features is enhanced. In other words, by introducing the channel interaction attention mechanism, the channel weights in the feature map are adaptively adjusted, and the model's ability to recognize key features is enhanced.

[0075] According to an embodiment of the present disclosure, in step S104, the second feature map is input into a domain classifier network for image-level feature alignment (whose loss function can be expressed as, for example, Figure 5 domain classification loss as shown) and an image-level multi-label classifier network for class-specific feature alignment (whose loss function can be expressed as Figure 5 The image-level classification loss shown in is used to obtain the third feature map after feature alignment.

[0076] That is, adversarial learning and image-level feature alignment can be achieved through step S104. Specifically, on the basis of redistributing the attention weight information, adversarial learning is used to further align the image-level feature distributions of the source domain and the target domain globally and locally, and a multi-label classifier is used to align category-specific features.

[0077] As an example, step S104 may further include steps S1041 and S1042:

[0078] In step S1041, a domain label (such as Figure 5 The domain labels Y shown here indicate the correspondence between each feature and the source domain and the target domain.

[0079] For example, a domain label is assigned to each extracted feature. Specifically, a one-hot label is used to represent the source domain of the input image. The label assignment formula is shown in the following formula (8):

[0080]

[0081] In step S1042, after performing the following processes S41 and S42 in parallel, a third feature map is obtained by combining all features after performing image-level feature alignment on both the deep features and the shallow features in the second feature map and performing category-specific feature alignment on the deep features in the second feature map:

[0082] Regarding processing S41, it includes: inputting the second feature map into a global domain classifier for deep features and a local domain classifier for shallow features, training the global domain classifier and the local domain classifier through an adversarial learning method and adjusting the training parameters through a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data.

[0083] For example, first, we design domain classifiers for global and local features respectively (e.g. Figure 5Here, for the global feature domain classifier, it can be composed of, for example, three groups of convolution (Conv2D), batch normalization (BatchNorm2D), nonlinear activation function ReLU and Dropout regularization layer, and then the spatial dimension of the feature map is pooled using the global average pooling operation to reduce the dimension of the feature, and the pooled features are flattened and resized, and the final domain classification result is output through the fully connected layer; for the local feature domain classifier, it can be calculated by three layers of convolution and Sigmoid activation function to obtain the domain classification result, and the network structure is simpler and has fewer calculation parameters.

[0084] Then, the input feature X is processed according to the forward propagation process of the domain classifier constructed above, and the classification loss between the classification prediction result and the one-hot label tensor constructed above is calculated. Here, the loss function for training the parameters of the global feature classifier is, for example, the Sigmoid focal loss function, which is used to process the class imbalance data set and improve the classification performance of the model for the minority class; the loss function for training the parameters of the local feature domain classifier is, for example, the mean square error loss (MSE), and the specific calculation formulas are shown in the following equations (9) and (10):

[0085]

[0086] Among them, y i is the true value, is the predicted value logits, N is the number of samples, α t is a balance parameter, γ is a focus parameter that adjusts the weights of easy-to-classify and difficult-to-classify samples, and p i It is expressed as the probability of the positive class predicted by the probability model obtained through Sigmoid transformation. In addition, Figure 5 The two gradient reversal layers (GRL) shown in Figure 1 update the feature extraction network to learn domain invariant features.

[0087] Regarding processing S42, it includes: inputting the deep features included in the first feature map into the image-level multi-label classifier network, training the image-level multi-label classifier network through a supervised learning method and adjusting the training parameters through an image-level classification loss function to achieve specific category feature alignment between the source domain data and the target domain data.

[0088] For example, image-level multi-label classifiers (e.g., Figure 5The construction process of the image-level classifier in the AIAM box is as follows: the deep features after attention redistribution are used as input, and the classification scores of each category are obtained through convolutional layers and global average pooling. The binary cross entropy loss is calculated based on the score and one-hot type image-level annotations. The category-specific features are learned under supervised training to achieve category feature alignment between different fields. The specific calculation formula is shown in the following formula (11):

[0089]

[0090] Among them, σ(·) represents the Sigmoid function, y i is the true value, are the predicted values ​​logits.

[0091] According to the embodiments of the present disclosure, adversarial learning strategies and domain classifiers are used to further align image-level features of the source domain and the target domain, and the model is optimized by focal loss and MSE loss to improve the generalization ability of the model for data in different fields, thereby improving the overall generalization ability of the model.

[0092] In addition, as an example, refer to Figure 5 , the module for implementing the CIS strategy can constitute an attention-based image-level alignment module AIAM together with the various classifiers in step S104.

[0093] According to an embodiment of the present disclosure, in step S105, the third feature map is input into a first prediction network for source domain instance-level feature alignment (whose loss function can be expressed as, for example, Figure 5 The source domain RPN loss and detection head loss shown in the figure) and the second prediction network for instance-level multi-label prediction in the target domain (whose loss function can be expressed as Figure 5 The instance-level classification loss shown in is used to obtain the fourth feature map.

[0094] As an example, step S105 may further include, after performing the following processes S51 and S52 in parallel, obtaining a fourth feature map by combining all features after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map:

[0095] Regarding processing S51, it includes: inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network through a supervised learning method to achieve instance-level feature alignment of the source domain data.

[0096] For example, process S51 may further include steps S511 to S513:

[0097] In step S511, the third feature map, the first image-level annotation information, and the instance-level annotation information are input into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information.

[0098] In step S512, a first loss function is calculated based on the first image-level annotation information and the instance-level annotation information as well as the first prediction information and the second prediction information.

[0099] In step S513, the network parameters of the first prediction network are adjusted according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0100] That is to say, source domain instance-level prediction alignment can be achieved by processing S51. Specifically, supervised learning (strong supervised learning) is performed using annotation information in the source domain (e.g., instance-level annotation information), candidate regions are generated through a region proposal network (RPN), and the classification loss and bounding box regression loss generated by the source domain image in the RPN and detection head network are calculated, such as using the SoftMax cross entropy loss function and the Smooth L1 loss function to optimize the model parameters, and instance-level feature alignment is achieved, thereby ensuring that the model has good prediction performance on the source domain data.

[0101] For example, a specific example of processing S51 may be as follows:

[0102] First, the candidate region is generated by RPN, and the RPN candidate box binary cross entropy classification loss and Smooth L1 regression loss are calculated.

[0103] Then, the detection head is used to perform category prediction and bounding box regression on the candidate region, and the classification loss and regression loss are minimized using SoftMax cross entropy and Smooth L1 respectively, and the network parameters are updated by back propagation. The calculation formulas of the two loss functions are shown in the following equations (12) and (13):

[0104]

[0105]

[0106] Among them, s represents all source domain training data, y i represents the true category label, represents the predicted label, λ is the L2 regularization term coefficient, θ represents the trainable parameter, and pre=(pre x , pre y , pre w , pre h ) represents the predicted bounding box coordinates, gt = (gt x ,gt y ,gtw ,gt h ) represents the true bounding box coordinates.

[0107] According to the above-mentioned embodiments of the present disclosure, in the source domain, the classification loss and regression loss generated by the RPN and the detection head are calculated using the existing annotation information to ensure the prediction accuracy of the model in the source domain.

[0108] Next, regarding processing S52, it includes: inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network through a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0109] For example, process S52 may further include steps S521 to S525:

[0110] In step S521, the deep features in the third feature map and the second image-level annotation information are input into the second prediction network to obtain third prediction information and fourth prediction information, wherein the third prediction information represents a confidence score that the target object is contained in the candidate box associated with the deep features, and the fourth prediction information represents a prediction score that the target object contained in the candidate box belongs to a specific category.

[0111] In step S522, based on the third prediction information and the fourth prediction information, a prediction score of a specific category at the instance level is reconstructed.

[0112] In step S523, final prediction information based on image-level category prediction is obtained based on the prediction score of the instance-level specific category, the third prediction information, and the fourth prediction information.

[0113] In step S524, a second loss function is calculated based on the second image-level annotation information and the final prediction information.

[0114] In step S525, the network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0115] That is, by processing S52, the target domain instance-level reconstruction and image-level prediction alignment can be achieved. Specifically, weakly supervised learning is performed on the target domain data, and the candidate box scores of the RPN and the classification prediction results of the detection head are aggregated through the instance-level multi-label prediction module to reconstruct the classification prediction matrix of the instance-level features, and the binary cross entropy loss is used for optimization.

[0116] Furthermore, on the target domain data, the RPN and the detection head are used to generate classification predictions of instance-level features, and the candidate target scores from the RPN are and the classification prediction matrix x∈ (in, is the total number of categories) together to achieve weakly supervised learning.

[0117] here, Indicates the predicted probability score of the target object in the candidate box generated by RPN, It indicates the probability score of the target object in the candidate box further predicted by the detection head belonging to each category. The two are combined with the multi-head classifier (for example, Figure 5 The image-level classifier in the I2ITM of the box) constitutes the instance-level multi-label prediction module (i.e., Figure 5 I2ITM in , and obtain the weighted adjusted category-specific prediction score

[0118] Specifically, for example, a specific example of processing S52 (specific flow is as follows Figure 4 As shown, no further details are given here Figure 4 The specific content of the

[0119] First, the new candidate matrix is ​​calculated based on the classification prediction matrix x of the detection head

[0120] Here, according to x n The confidence score of n Assign to a specific target class along the row: if x n The category index of the maximum value is i, then the score is o n Assign to the i-th category; if x n The minimum classification index is j, then the score is o n The opposite number of is assigned to the jth class; the predicted scores of all other classes are assigned to 0; finally, the target score matrix for a specific class is obtained Among them, categories with high classification probability obtain high confidence, categories with low probability obtain low confidence, and the remaining categories are all set to 0, thus obtaining a category probability distribution matrix with strong category information indication.

[0121] Then, the SoftMax function is used to weight the initial score matrix x and obtain the new category-specific prediction matrix Normalize along the rows and columns, calculate the weighted recommendation scores of the two, and finally add them in SoftMax(x) and Perform element-wise multiplication between them to obtain the weighted adjusted prediction matrix The final image-level prediction result P is obtained by summing them up. Formally speaking, the image-level prediction aggregation can be expressed as the following formula (14):

[0122]

[0123] in, is the predicted probability of category c (indicating whether the current image contains an object of category c), σ row (·) and σ col (·) represents the SoftMax operation along the rows and columns respectively, and (·) represents the element-wise product operation.

[0124] Finally, under the premise of image-level multi-label prediction and one-hot label encoding, binary cross-entropy loss is used for training and back-propagation optimization of the model to reduce the domain differences of instance-level features through weakly supervised training.

[0125] According to the above embodiments of the present disclosure, in the target domain, weakly supervised learning is achieved by combining the predictions of RPN and detection head with the instance-level multi-label prediction module, and the impact of domain differences is effectively reduced through binary cross entropy loss optimization.

[0126] According to an embodiment of the present disclosure, in step S106, model parameters of the cross-domain object detection model are adjusted according to the source domain data, the target domain data and the first to fourth feature maps to obtain a trained cross-domain object detection model.

[0127] As an example, the following can be obtained based on the source domain data, the target domain data, and the first to fourth feature maps: Figure 5 The five losses shown are used to adjust the parameters of the entire model by minimizing the sum of these five losses L. For example, The expression of can be shown as the following formula (15):

[0128]

[0129] Among them, the loss weight λ dc , ic and λ cls are preset parameters, for example, they can be set to 1, 0.1 and 1 respectively.

[0130] According to the embodiments of the present disclosure, through the above-mentioned training method of the present disclosure, weakly supervised cross-domain target detection that integrates adversarial learning and self-perceived hierarchical alignment is realized, and the target detection technology in different cross-domain scenarios of the present disclosure is utilized to solve the problem of a significant decrease in model detection performance caused by domain differences.

[0131] In addition, by utilizing theoretical models such as deep learning methods, adversarial learning methods, and attention mechanisms, and integrating cross-domain, adversarial feature learning, and holistic to hierarchical alignment, an end-to-end cross-domain target detection framework is constructed, thereby improving the accuracy of target detection across domains in different scenarios, and can further serve various application scenarios such as traffic scenarios and virtual reality scenarios.

[0132] Here, it should be noted that the above Figure 1 The description of each step is merely exemplary, and the steps in the method according to the present disclosure are not limited thereto.

[0133] In addition, the present disclosure provides a cross-domain target detection method for image data. The cross-domain target detection method includes: obtaining image data to be detected; inputting the image data to be detected into a cross-domain target detection model for the image to obtain corresponding image detection data. Here, the cross-domain target detection model is trained using the training method described above.

[0134] Next, refer to Figure 6 To describe a training device for a cross-domain object detection model for images according to an embodiment of the present disclosure.

[0135] Figure 6 is a block diagram showing a training device 600 for a cross-domain object detection model for an image according to an embodiment of the present disclosure. For example, the cross-domain object detection model includes a deep learning network, a domain classifier network, an image-level multi-label classifier network, a first prediction network, and a second prediction network.

[0136] Reference Figure 6 According to an embodiment of the present disclosure, a training device 600 for a cross-domain target detection model for an image may include a data acquisition module 610, a feature extraction module 620, a weight adjustment module 630, a first feature alignment module 640, a second feature alignment module 650 and a parameter adjustment module 660.

[0137] According to an embodiment of the present disclosure, the data acquisition module 610 may execute: acquiring source domain data and target domain data. Here, the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data.

[0138] In addition, as an example, the source domain image annotation data may include first image-level annotation information and instance-level annotation information, and the target domain image annotation data may include second image-level annotation information. That is, the target domain image annotation data does not include instance-level annotation information, but the present disclosure is not limited thereto, and may also include less instance-level annotation information that is insufficient for performing the supervised learning method.

[0139] According to an embodiment of the present disclosure, the feature extraction module 620 may execute: inputting source domain data and target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image.

[0140] According to an embodiment of the present disclosure, the weight adjustment module 630 can execute: based on a cross-channel cross-attention strategy, adjust the attention weight in the first feature map to obtain the adjusted attention weight, and use the first feature map with the adjusted attention weight as the second feature map.

[0141] As an example, the above operation performed by the weight adjustment module 630 may further include: for each feature of the deep features and shallow features included in the first feature map, adjusting the attention weight of each channel according to the above formulas (3) to (5) to obtain an adjusted attention weight.

[0142] According to an embodiment of the present disclosure, the first feature alignment module 640 may execute: inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for category-specific feature alignment to obtain a third feature map after feature alignment.

[0143] As an example, the above operations performed by the first feature alignment module 640 may further include the following operations 641) and 642):

[0144] In operation 641), a domain label is added to each feature in the second feature map, where the domain label indicates the correspondence between each feature and the source domain and the target domain.

[0145] In operation 642), after performing the following processes (1) and (2) in parallel, a third feature map is obtained by combining all features after performing image-level feature alignment on both the deep features and the shallow features in the second feature map and performing category-specific feature alignment on the deep features in the second feature map:

[0146] (1) The second feature map is input into a global domain classifier for deep features and a local domain classifier for shallow features, the global domain classifier and the local domain classifier are trained by an adversarial learning method, and the training parameters are adjusted by a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data.

[0147] (2) The deep features included in the first feature map are input into an image-level multi-label classifier network, the image-level multi-label classifier network is trained by a supervised learning method, and the training parameters are adjusted by an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

[0148] According to an embodiment of the present disclosure, the second feature alignment module 650 may execute: inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map.

[0149] As an example, the above operation to be performed by the second feature alignment module 650 may further include: after performing the following processes 651) and 652) in parallel, by combining all features after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map, a fourth feature map is obtained:

[0150] For processing 651), the third feature map and the source domain data are input into a first prediction network for source domain instance-level feature alignment, and the first prediction network is trained by a supervised learning method to achieve instance-level feature alignment of the source domain data.

[0151] For example, the above process 651) may further include operations 6511) and 6512):

[0152] In operation 6511), the third feature map, the first image-level annotation information, and the instance-level annotation information are input into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information.

[0153] In operation 6512), a first loss function is calculated based on the first image-level annotation information and the instance-level annotation information as well as the first prediction information and the second prediction information, and a network parameter of the first prediction network is adjusted according to the first loss function to achieve instance-level feature alignment of the source domain data.

[0154] For processing 652), the third feature map and the target domain data are input into a second prediction network for target domain instance-level category prediction, and the second prediction network is trained by a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0155] For example, the above process 652) may further include operations 6521) and 6524):

[0156] In operation 6521), the deep features in the third feature map and the second image-level annotation information are input into the second prediction network to obtain third prediction information and fourth prediction information.

[0157] Here, the third prediction information represents a confidence score that the target object is contained in the candidate frame associated with the deep feature, and the fourth prediction information represents a prediction score that the target object contained in the candidate frame belongs to a specific category.

[0158] In operation 6522), based on the third prediction information and the fourth prediction information, a prediction score for a specific category at the instance level is reconstructed.

[0159] In operation 6523), based on the prediction score of the instance-level specific category, the third prediction information and the fourth prediction information, final prediction information based on image-level category prediction is obtained.

[0160] In operation 6524), based on the second image-level annotation information and the final prediction information, a second loss function is calculated, and network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

[0161] According to an embodiment of the present disclosure, the parameter adjustment module 660 may be executed to adjust the model parameters of the cross-domain target detection model according to the source domain data and the target domain data and the first to fourth feature maps to obtain a trained cross-domain target detection model.

[0162] It should be noted that the operations performed on the above-mentioned structural frames can be referred to in Figure 1 The related contents described are similar and will not be repeated here.

[0163] Figure 7 is a block diagram illustrating a computing device 700 according to an embodiment of the present disclosure.

[0164] Reference Figure 7 According to an embodiment of the present disclosure, the computing device 700 may include a processor 710 and a memory 720. The processor 710 may include (but is not limited to) a central processing unit (CPU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a system on a chip (SoC), a microprocessor, an application-specific integrated circuit (ASIC), etc. The memory 720 may store computer-executable instructions to be executed by the processor 710. The memory 720 includes a high-speed random access memory and / or a non-volatile computer-readable storage medium. When the processor 710 executes the computer-executable instructions stored in the memory 720, the training method and device for the cross-domain target detection model for the image and the cross-domain target detection method as described above may be implemented.

[0165] According to an embodiment of the present disclosure, the training method and device of the cross-domain target detection model for images and the cross-domain target detection method can be written as a computer program / instruction to form a computer program product and stored on a computer-readable storage medium. When the computer program / instruction is executed by a processor, the training method and device of the cross-domain target detection model for images as described above and the cross-domain target detection method can be implemented. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device / server, the electronic device / server is enabled to execute the training method and device of the cross-domain target detection model for images as described above and the cross-domain target detection method. Examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer program and any associated data, data files and data structures in a non-transitory manner and provide the computer program and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. In one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0166] According to the training method and device of the cross-domain target detection model for images and the cross-domain target detection method of the embodiments of the present invention, a cross-domain weakly supervised target detection algorithm for image data is implemented by constructing a cross-domain strategy that integrates adversarial learning and self-perceived hierarchical alignment, thereby improving the accuracy of image detection tasks in cross-domain scenarios.

[0167] In addition, the training method and device of the cross-domain target detection model for images and the cross-domain target detection method according to the embodiments of the present disclosure solve the problem of a significant decrease in model detection performance caused by domain differences.

[0168] In addition, for more fine-grained target detection tasks, features with specific category information and high recognition usually play a key role in predicting output, while suppressing model overfitting caused by non-correlated areas. Therefore, the above-mentioned method disclosed in the present invention can more effectively complete such more fine-grained target detection tasks.

[0169] Although some embodiments of the present disclosure have been disclosed and described, it will be appreciated by those skilled in the art that modifications and variations may be made to the embodiments without departing from the concept and spirit of the present disclosure, the scope of which is defined by the claims and their equivalents.

Claims

1. A training method for a cross-domain object detection model for an image, characterized in that: The training method comprises: Acquire source domain data and target domain data, wherein the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data; Inputting the source domain data and the target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; Based on a cross-channel cross attention strategy, adjusting the attention weight in the first feature map to obtain an adjusted attention weight, and using the first feature map with the adjusted attention weight as the second feature map; Inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for class-specific feature alignment to obtain a third feature map after feature alignment; Inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; According to the source domain data, the target domain data and the first to fourth feature maps, the model parameters of the cross-domain target detection model are adjusted to obtain a trained cross-domain target detection model. Among them, the cross-domain target detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network and the second prediction network.

2. The training method according to claim 1, characterized in that: The step of adjusting the attention weight in the first feature map based on the cross-channel cross attention strategy to obtain the adjusted attention weight comprises: For each feature in the deep features and shallow features included in the first feature map, the attention weight of each channel is adjusted according to the following formula to obtain the adjusted attention weight: g=C1D {k} (g(x)) Among them, g represents the attention weight, C1D represents fast one-dimensional convolution, k represents the number of adjacent channels and the expression of k is: g(χ) represents the global average pooling of the channel dimension of the image data and the expression of g(χ) is: Among them, X represents the channel dimension, γ and b are preset parameter values, || odd represents the nearest odd number, The tensor data representing the image data, W and H represent the width and height of the tensor respectively, i is a natural number ranging from 1 to W, and j is a natural number ranging from 1 to H.

3. The training method according to claim 1, characterized in that: The step of inputting the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for class-specific feature alignment to obtain a third feature map after feature alignment comprises: Adding a domain label to each feature in the second feature map, where the domain label indicates a corresponding relationship between each feature and a source domain and a target domain; The third feature map is obtained by combining all features after performing image-level feature alignment on both the deep features and the shallow features in the second feature map and performing category-specific feature alignment on the deep features in the second feature map after performing the following processing in a parallel manner: Inputting the second feature map into a global domain classifier for deep features and a local domain classifier for shallow features, training the global domain classifier and the local domain classifier by an adversarial learning method and adjusting training parameters by a domain classification loss function to achieve image-level feature alignment between source domain data and target domain data; and The deep features included in the first feature map are input into the image-level multi-label classifier network, the image-level multi-label classifier network is trained by a supervised learning method and the training parameters are adjusted by an image-level classification loss function to achieve specific category feature alignment between source domain data and target domain data.

4. The training method according to claim 1, characterized in that: The step of inputting the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map comprises: After performing the following processing in a parallel manner, the fourth feature map is obtained by combining all features after performing instance-level feature alignment on features associated with source domain data in the third feature map and performing instance-level feature alignment based on image-level category prediction on features associated with target domain data in the third feature map: Inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network by a supervised learning method to achieve instance-level feature alignment of the source domain data; The third feature map and the target domain data are input into a second prediction network for target domain instance-level category prediction, and the second prediction network is trained by a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction.

5. The training method according to claim 4, characterized in that: The source domain image annotation data includes first image-level annotation information and instance-level annotation information, and the step of inputting the third feature map and the source domain data into a first prediction network for source domain instance-level feature alignment, and training the first prediction network by a supervised learning method to achieve instance-level feature alignment of the source domain data includes: Inputting the third feature map, the first image-level annotation information, and the instance-level annotation information into the first prediction network to obtain first prediction information corresponding to the first image-level annotation information and second prediction information corresponding to the instance-level annotation information; Based on the first image-level annotation information and the instance-level annotation information and the first prediction information and the second prediction information, a first loss function is calculated, and network parameters of the first prediction network are adjusted according to the first loss function to achieve instance-level feature alignment of source domain data.

6. The training method according to claim 4, characterized in that: The target domain image annotation data includes second image-level annotation information, and the step of inputting the third feature map and the target domain data into a second prediction network for target domain instance-level category prediction, and training the second prediction network by a weakly supervised learning method to achieve instance-level feature alignment of the target domain data based on image-level category prediction includes: Inputting the deep features in the third feature map and the second image-level annotation information into the second prediction network to obtain third prediction information and fourth prediction information, wherein the third prediction information indicates a confidence score that the target object is contained in the candidate box associated with the deep features, and the fourth prediction information indicates a prediction score that the target object contained in the candidate box belongs to a specific category; reconstructing a prediction score of a specific category at the instance level based on the third prediction information and the fourth prediction information; Obtaining final prediction information based on image-level category prediction based on the instance-level specific category prediction score, the third prediction information, and the fourth prediction information; Based on the second image-level annotation information and the final prediction information, a second loss function is calculated, and network parameters of the second prediction network are adjusted according to the second loss function to achieve instance-level feature alignment of target domain data based on image-level category prediction.

7. A training device for a cross-domain object detection model for an image, characterized in that: The training device comprises: A data acquisition module is configured to: acquire source domain data and target domain data, wherein the source domain data includes source domain image data and source domain image annotation data, and the target domain data includes target domain image data and target domain image annotation data; A feature extraction module is configured to: input the source domain data and the target domain data into a pre-trained deep learning network to obtain a first feature map for both the source domain image and the target domain image; A weight adjustment module is configured to: adjust the attention weight in the first feature map based on a cross-channel cross attention strategy to obtain an adjusted attention weight, and use the first feature map with the adjusted attention weight as the second feature map; A first feature alignment module is configured to: input the second feature map into a domain classifier network for image-level feature alignment and an image-level multi-label classifier network for specific category feature alignment to obtain a third feature map after feature alignment; The second feature alignment module is configured to: input the third feature map into a first prediction network for source domain instance-level feature alignment and a second prediction network for target domain instance-level multi-label prediction to obtain a fourth feature map; A parameter adjustment module is configured to: adjust the model parameters of the cross-domain target detection model according to the source domain data, the target domain data and the first feature map to the fourth feature map to obtain a trained cross-domain target detection model, Among them, the cross-domain target detection model includes the deep learning network, the domain classifier network, the image-level multi-label classifier network, the first prediction network and the second prediction network.

8. A cross-domain target detection method for image data, characterized in that: The cross-domain target detection method comprises: Acquire image data to be detected; The image data to be detected is input into a cross-domain target detection model for images to obtain corresponding image detection data. The cross-domain target detection model is trained using the training method described in any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product includes a computer program / instructions, which, when executed by a processor, implements the training method for a cross-domain target detection model for images as described in any one of claims 1 to 6 and the cross-domain target detection method for image data as described in claim 8.

10. A computing device, characterized in that The computing device includes: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to execute the training method for a cross-domain target detection model for images as described in any one of claims 1 to 6 and the cross-domain target detection method for image data as described in claim 8.

Images