Remote sensing cross-domain target detection model training method and device, and detection method

Abstract

The application discloses a training method and device of a target detection model for remote sensing cross-domain and a detection method, and belongs to the technical field of remote sensing image detection; the method comprises the following steps: obtaining a source domain remote sensing image with labels and a target domain remote sensing image without labels; using a shared feature extraction network of the target detection model to extract detection feature maps of multiple scales from the source domain remote sensing image and the target domain remote sensing image, so as to determine original amplitude spectra, original phase spectra and category composition embeddings corresponding to the multiple scales; generating channel modulation factors corresponding to the multiple scales according to learnable scale embedding parameters and the category composition embeddings corresponding to the multiple scales, so as to modulate amplitude values of low-frequency regions in the original amplitude spectra corresponding to the multiple scales, and obtain modulated amplitude spectra corresponding to the multiple scales; and determining modulated spatial features corresponding to the multiple scales according to the modulated amplitude spectra corresponding to the multiple scales and the original phase spectra corresponding to the multiple scales, so as to optimize the learnable scale embedding parameters and parameters of the target detection model.

Description

Technical Field

[0001] This application relates to the field of remote sensing image detection technology, specifically to a training method, apparatus, and detection method for a remote sensing cross-domain target detection model. Background Technology

[0002] Remote sensing image target detection models often need to be migrated and used under different satellite platforms, sensor payloads, and imaging conditions in practical deployments. Differences in spectral response, spatial resolution, and imaging geometry between platforms cause significant variations in brightness, texture, and scale distribution of the same type of target in the source and target domains. Re-labeling training data for each new target domain is costly and time-consuming. Therefore, unsupervised adaptive target detection has become an important technical approach in remote sensing intelligent interpretation.

[0003] Unsupervised domain adaptive object detection schemes in related technologies mainly include: pixel-level alignment of feature maps and instance features in the spatial domain, image-level domain adaptation strategies based on frequency domain transfer or style-content decoupling, and feature enhancement mechanisms combining spatial-frequency dual-domain information. Among these, the scheme that aligns the entire feature map, region proposal, or instance features as a whole can alleviate the appearance differences between the source and target domains, but it is prone to destroying the target's geometric structure while eliminating appearance differences. Summary of the Invention

[0004] This application provides a training method, apparatus, and detection method for a remote sensing cross-domain target detection model, which addresses the problem that unsupervised domain adaptive target detection methods in related technologies tend to destroy the target's geometric structure while eliminating appearance differences.

[0005] To achieve the above objectives, the embodiments of this application disclose the following technical solutions: In a first aspect, embodiments of this application provide a training method for a remote sensing cross-domain target detection model, comprising the following steps: Acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images; By utilizing the shared feature extraction network of the target detection model, detection feature maps at multiple scales are extracted from source domain remote sensing images and target domain remote sensing images; Based on the detection feature maps at each scale, determine the original amplitude spectrum, original phase spectrum, and category composition embedding corresponding to each scale; The embedding is composed of the learnable scale embedding parameters and categories corresponding to each scale, and the channel modulation factor corresponding to each scale is generated. Based on the channel modulation factor corresponding to each scale, the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale is modulated to obtain the modulated amplitude spectrum corresponding to each scale. The modulated spatial features corresponding to each scale are obtained by performing inverse frequency domain transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale. Based on the modulated spatial features corresponding to each scale, optimize the learnable scale embedding parameters and the parameters of the target detection model.

[0006] In some possible implementations of the first aspect, the step of determining the original amplitude spectrum and original phase spectrum corresponding to each scale based on the detection feature maps at each scale includes: A frequency domain transformation is performed on the detection feature maps at each scale to obtain the frequency domain complex spectrum corresponding to each scale; Amplitude and phase separation is performed on the frequency domain complex spectrum corresponding to each scale to obtain the original amplitude spectrum and the original phase spectrum corresponding to each scale.

[0007] In some possible implementations of the first aspect, the annotation of the source domain remote sensing image includes bounding boxes, and the step of determining the category composition embedding corresponding to each scale based on the detection feature maps at each scale includes: From the detection feature maps at various scales, extract the spatial features within the bounding box regions of each category at each scale; For each category at each scale, spatial pooling is performed on the spatial features within the corresponding bounding box area to obtain temporary category prototypes for each category at each scale. Based on the temporary category prototypes corresponding to each category at each scale, the source domain feature prototype library for each category at each scale is updated by moving average. Based on the detection feature maps at each scale and the updated source domain feature prototype library, the category composition embedding at each scale is determined.

[0008] In some possible implementations of the first aspect, the source domain feature prototype library for each scale and category is updated according to the following formula: ; In the formula, For the updated version The first scale, the first Source domain feature prototypes for each category, For prototype update coefficients, This is a temporary category prototype.

[0009] In some possible implementations of the first aspect, the step of determining the category composition embedding at each scale based on the detection feature maps at each scale and the updated source domain feature prototype library includes: For each scale, calculate the feature similarity between the features at each spatial location in the detected feature map and the source domain feature prototypes of each category in the source domain feature prototype library; Spatial maximum aggregation is performed on the non-negative responses of feature similarity for each category to obtain aggregated similarity for each category; The aggregated similarity of each category is normalized along the category dimension, and the normalized probability distribution vector is determined as the category composition embedding corresponding to each scale.

[0010] In some possible implementations of the first aspect, the feature similarity is cosine similarity, and the feature similarity between the features at each spatial location and the source domain feature prototypes of each category is determined by the following formula: ; In the formula, For the first The first scale The spatial location and the first Feature similarity between source domain feature prototypes of each category For the first The first scale Characteristics of a spatial location For the first The first scale, the first The prototype of source domain features for each category.

[0011] In some possible implementations of the first aspect, the category composition embedding corresponding to each scale is determined by the following formula: ; In the formula, For the first The categories corresponding to each scale form the embedding. For feature similarity, Total number of categories; Indicates spatial location index, This indicates taking the maximum value across all spatial locations. Indicates a corrected linear unit. This indicates normalization along the category dimension.

[0012] In some possible implementations of the first aspect, the step of generating the channel modulation factor corresponding to each scale by composing an embedding based on the learnable scale embedding parameters and category corresponding to each scale includes: For each scale, the learnable scale embedding parameters and categories corresponding to the scale are combined into an embedding fusion vector to form a scale category condition vector; Input the scale category condition vector into the condition vector mapping network to obtain the mapping output; An activation function is applied to the mapped output to constrain the numerical range, thus obtaining the channel modulation factor corresponding to each scale.

[0013] In some possible implementations of the first aspect, the channel modulation factor corresponding to each scale is determined by the following formula: ; In the formula, For the first The channel modulation factor corresponding to each scale This is a scale-category condition vector. For conditional vector mapping networks, This is the activation function.

[0014] In some possible implementations of the first aspect, the step of modulating the amplitude values ​​of the low-frequency region in the original amplitude spectrum corresponding to each scale according to the channel modulation factor corresponding to each scale to obtain the modulated amplitude spectrum corresponding to each scale includes: For each scale, a low-frequency smoothing mask is constructed based on the frequency domain size of the original amplitude spectrum. The low-frequency smoothing mask takes a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region. For each scale, the channel modulation factor corresponding to the scale is broadcast to the same shape as the original amplitude spectrum, and the modulation region is controlled according to the low-frequency smoothing mask to obtain the modulated amplitude spectrum corresponding to each scale.

[0015] In some possible implementations of the first aspect, the low-frequency smoothing mask is determined by the following formula: ; In the formula, For the first Each scale at frequency position Low-frequency smoothing mask value at the location, This is the normalized distance from the frequency position to the DC component position. For the first Low-frequency bandwidth parameters at each scale.

[0016] In some possible implementations of the first aspect, the modulated amplitude spectrum corresponding to each scale is determined by the following formula: ; In the formula, For the first The modulated amplitude spectrum corresponding to each scale. The original amplitude spectrum, For low-frequency smoothing masks, For channel modulation factor, This indicates element-wise multiplication.

[0017] In some possible implementations of the first aspect, the step of performing inverse frequency domain transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale to obtain the modulated spatial features corresponding to each scale includes: For each scale, the frequency domain complex spectrum is reconstructed based on the modulated amplitude spectrum and the original phase spectrum corresponding to the scale. The reconstructed frequency domain complex spectrum is subjected to inverse frequency domain transformation to obtain the modulated spatial features corresponding to each scale.

[0018] In some possible implementations of the first aspect, the modulated spatial features corresponding to each scale are determined by the following formula: ; In the formula, For the first Modulated spatial features corresponding to each scale The amplitude spectrum after modulation. The original phase spectrum, The imaginary unit, It is the inverse two-dimensional real fast Fourier transform.

[0019] In some possible implementations of the first aspect, the step of optimizing the learnable scale embedding parameters and the parameters of the target detection model based on the modulated spatial features corresponding to each scale includes: The modulated spatial features corresponding to each scale are fused with the detection feature maps corresponding to each scale using a residual reconstruction method to obtain the training-period adversarial features corresponding to each scale. The adversarial features corresponding to each scale during the training period are input into the domain discriminator, and the domain confusion constraint is applied to the backbone network and feature projection module of the shared feature extraction network through the gradient inversion layer to obtain the domain adversarial loss corresponding to each scale. Based on the bounding boxes and category labels of the source domain remote sensing images, determine the detection supervision loss; The total loss is the weighted sum of the detection supervision loss and the domain adversarial loss corresponding to each scale. The learnable scale embedding parameters and the parameters of the object detection model are iteratively updated until the total loss converges or the preset training rounds are reached. During the training process, the main detection path of the object detection model takes the detection feature map directly provided by the shared feature extraction network as input.

[0020] In some possible implementations of the first aspect, the training-period adversarial features corresponding to each scale are determined by the following formula: ; In the formula, For the first The training-period adversarial features corresponding to each scale To detect feature maps, For modulated spatial features, Weights are reconstructed from learnable residuals.

[0021] In some possible implementations of the first aspect, the total loss is determined by the following formula: ; In the formula, For the total loss, To detect and monitor losses, For the first Domain adversarial loss corresponding to each scale For domain adversarial loss weights, This represents the total number of scales.

[0022] Secondly, embodiments of this application provide a training apparatus for a remote sensing cross-domain target detection model, comprising: The data acquisition module is used to acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images; The multi-scale feature extraction module is used to extract detection feature maps at multiple scales from source domain remote sensing images and target domain remote sensing images by utilizing the shared feature extraction network of the target detection model. The first generation module is used to determine the original amplitude spectrum and original phase spectrum corresponding to each scale, as well as the category composition embedding, based on the detection feature maps at each scale. The second generation module is used to generate the channel modulation factor corresponding to each scale by forming an embedding based on the learnable scale embedding parameters and categories corresponding to each scale. The low-frequency amplitude modulation module is used to modulate the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale according to the channel modulation factor corresponding to each scale, so as to obtain the modulated amplitude spectrum corresponding to each scale. The spatial feature reconstruction module is used to perform inverse frequency domain transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale to obtain the modulated spatial features corresponding to each scale. The parameter optimization module is used to optimize the learnable scale embedding parameters and the parameters of the target detection model based on the modulated spatial features corresponding to each scale.

[0023] Thirdly, embodiments of this application also provide a cross-domain remote sensing image detection method, including: Acquire the remote sensing image to be detected; The shared feature extraction network of the remote sensing cross-domain target detection model is used to extract detection feature maps of multiple scales from the remote sensing image to be detected; wherein, the remote sensing cross-domain target detection model is trained by the training method of the remote sensing cross-domain target detection model described in any of the embodiments of the first aspect above; The detection feature maps at each scale are input into the main detection path of the remote sensing cross-domain target detection model to obtain the target detection results at each scale.

[0024] In some embodiments of the third aspect, the remote sensing cross-domain target detection model retains only the aforementioned shared feature extraction network and detection main path during the inference phase, without enabling frequency domain transformation, frequency domain modulation, frequency domain inverse transformation, domain discriminator, and gradient inversion layer.

[0025] Fourthly, embodiments of this application provide an electronic device, including: One or more processing devices; A storage device on which one or more programs are stored; When one or more programs are executed by one or more processing devices, the one or more processing devices implement the method described in any of the first or third aspects above.

[0026] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processing device, implements the method described in any one of the first or third aspects described above.

[0027] One or more technical solutions provided in the first to fifth aspects of the embodiments of this application have at least the following technical effects: 1. By determining the original amplitude spectrum, original phase spectrum, and category composition embedding based on the detection feature map, the amplitude information, phase information, and category semantic information of the detection feature map can be separated. In this way, a channel modulation factor can be generated jointly by the learnable scale embedding parameters and the category composition embedding. Applying the channel modulation factor to the low-frequency region of the original amplitude spectrum allows the modulation intensity to simultaneously match the current scale level and the category composition in the current feature, while the modulation operation is confined to the low-frequency amplitude component range.

[0028] Building upon this, by performing an inverse frequency domain transformation between the modulated amplitude spectrum and the original phase spectrum to obtain the modulated spatial features, the original phase spectrum remains numerically unchanged throughout the entire process from determination to inverse transformation. Therefore, the modulated spatial features possess the characteristic of adaptively adjusting the amplitude distribution while fully preserving spatial structure information. Consequently, when the modulated spatial features are used to simultaneously optimize the learnable scale embedding parameters and the parameters of the target detection model, allowing the modulation strategy to converge collaboratively with the detection task objective during training, the target detection model can automatically adapt to the distribution differences between the source and target domains. This ensures that the trained target detection model eliminates the appearance differences between the source and target domains without damaging the target's geometry, thereby improving the accuracy of cross-domain target detection in remote sensing.

[0029] 2. By spatially pooling and aggregating the spatial features within the bounding boxes of various categories at different scales, temporary category prototypes for each category are obtained. These temporary category prototypes are then fused into the source domain feature prototype library using a moving average method. This allows the source domain feature prototype library to gradually accumulate stable feature representations of each category at each scale during training. Specifically, spatial pooling can compress features from multiple spatial locations within the bounding box into a single prototype vector, eliminating the interference of individual abnormal pixels within the box on prototype estimation. Moving average updates allow new batches of feature information to be integrated into the prototype library at a controlled rate, avoiding drastic fluctuations in prototypes caused by accidental noise in a single batch of samples. Thus, the source domain feature prototype library can provide a stable and reliable reference basis for subsequent category composition estimation, ensuring that the category composition embedding calculated based on the prototype library accurately reflects the category distribution of the current detection feature map, thereby providing a reliable category condition input for the channel modulation factor.

[0030] 3. By calculating the feature similarity between features at each spatial location in the detection feature map and source domain feature prototypes of each category in the source domain feature prototype library, and then performing spatial maximum aggregation after taking the non-negative response of the feature similarity for each category, followed by normalization along the category dimension, it is possible to generate category composition embeddings reflecting the presence intensity of each category of target in the detection feature map at each scale without relying on manual annotation of the target domain. Specifically, spatial maximum aggregation can use the most significant response position of each category in the entire detection feature map as the criterion for the presence or absence of that category, thus avoiding the need for explicit detection of the target spatial range; non-negative response filtering can eliminate the interference of irrelevant feature vectors with directions opposite to those of the source domain feature prototypes; normalization can ensure that the category composition embeddings of different scales and batches maintain a consistent numerical scale. Thus, meaningful category composition embeddings can also be obtained for remote sensing images of the target domain, allowing the generation of channel modulation factors to be differentiated according to the implicit category distribution in the detection feature map.

[0031] 4. By constructing a low-frequency smoothing mask based on the frequency domain dimensions of the original amplitude spectrum, and ensuring that the low-frequency smoothing mask has a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region, a continuous modulation weight transition can be formed between the low-frequency and high-frequency regions. Based on this, the channel modulation factors are broadcast to the same shape as the original amplitude spectrum, and the modulation region is controlled according to the low-frequency smoothing mask. This allows the modulation intensity to change smoothly and gradually from low to high frequencies, rather than abruptly changing at a certain frequency. Therefore, it avoids the ringing effect or characteristic instability caused by introducing abrupt boundary changes in the frequency domain, ensuring that the spectral quality of the modulated amplitude spectrum is not affected by artifacts.

[0032] 5. By fusing the modulated spatial features with the detection feature map using residual reconstruction, adversarial features are obtained during the training phase. These adversarial features are then input into the domain discriminator and subjected to domain confusion constraints through a gradient inversion layer. This allows the frequency domain modulation branch to indirectly optimize the parameters of the shared feature extraction network only during the training phase through adversarial gradients, without directly replacing the input features of the main detection path. Simultaneously, the main detection path uses the detection feature map directly provided by the shared feature extraction network as input, ensuring that the spatial localization capability of the detection branch is not directly affected by the modulation features. This maintains the stability of the detection feature distribution and avoids interference with the spatial localization accuracy of target detection during the frequency domain alignment process.

[0033] 6. By separating the amplitude and phase spectra through frequency domain transformation, selectively modulating only low-frequency amplitudes, using scale embedding and category-based embedding to jointly determine the modulation intensity, and decoupling the modulated spatial features from the main detection path and using them only for the adversarial branch during training, the differences in low-frequency energy distribution between source and target domain detection features can be gradually reduced during training, while protecting the spatial structure and high-frequency detail information carried by the phase spectrum. After the source and target domain detection features become more consistent in their low-frequency amplitude distribution, the features extracted by the target detection model under different sensor platforms, spatial resolutions, and imaging conditions are more consistent. This results in higher detection stability, lower false negative rate, and lower background false positive rate in the target domain, enhancing the cross-domain adaptability of the target detection model under conditions without target domain annotation.

[0034] 7. By enabling the model to enter the inference phase, frequency domain transformation, frequency domain modulation, inverse frequency domain transformation, domain discriminator and gradient inversion layer are no longer enabled. Therefore, there is no need to perform FFT calculation, inverse FFT calculation and domain discriminator discrimination. No additional computational overhead is generated in the deployment phase, and the complexity of the model structure is not increased. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a flowchart illustrating a training method for a remote sensing cross-domain target detection model provided in some embodiments of this application.

[0037] Figure 2 This is a schematic diagram of the structure of a training device for a remote sensing cross-domain target detection model provided in some embodiments of this application.

[0038] Figure 3This is a schematic diagram of the structure of an electronic device suitable for implementing a training method for a remote sensing cross-domain target detection model, as provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the embodiments of this application, the terms "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0041] In the description of the embodiments of this application, the term "multiple" means two or more.

[0042] In the description of the embodiments of this application, the term "and / or" refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship.

[0043] In the description of embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0044] As mentioned in the background technology section above, remote sensing target detection has been widely used in scenarios such as land and resources surveys, refined urban management, disaster emergency assessment, maritime supervision, transportation infrastructure inspection, airport and port monitoring, and military reconnaissance.

[0045] In these applications, detection models typically need to be migrated between different satellite platforms, payloads, UAV platforms, seasons, and shooting angles. Due to differences in spectral response, ground sampling distance, spatial resolution, imaging height, viewing angle, compression processing, image post-processing, and ground background across platforms, the same type of target can exhibit significantly different brightness, texture, edge sharpness, and scale distribution in the source and target domains. The source and target domains can originate from different satellite platforms, UAV platforms, or imaging conditions. For example, models trained using data from sunny days or specific seasons can be adapted to images with shadows, haze, different solar altitude angles, or different surface backgrounds. Alternatively, they can be deployed across domains in scenarios with sparse targets and complex backgrounds, such as ports, airports, densely populated areas, and areas with ships and storage tanks at sea, while sharing the same set of target categories. Re-labeling training data for each target domain is costly and time-consuming; therefore, unsupervised adaptive target detection with labeled source domains and unlabeled target domains has become an important technical approach in remote sensing intelligent interpretation.

[0046] Unsupervised domain adaptive object detection schemes in related technologies mainly include: pixel-level alignment of feature maps and instance features in the spatial domain, image-level domain adaptation strategies based on frequency domain transfer or style-content decoupling, and feature enhancement mechanisms combining spatial-frequency dual-domain information. Among these, the scheme that aligns the entire feature map, region proposal, or instance features as a whole can alleviate the appearance differences between the source and target domains, but it does not distinguish between low-frequency amplitude, phase, and high-frequency details, which can easily destroy the target's geometric structure while eliminating appearance differences.

[0047] To address the aforementioned issues, this application provides a training method for a remote sensing cross-domain target detection model. This method can be executed by an execution entity such as a processor, server, or electronic device. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps:

[0048] S101, acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images.

[0049] For example, a remote sensing image with bounding boxes and category labels is read from the source domain (i.e., the source domain remote sensing image), and an unlabeled remote sensing image is read from the target domain (i.e., the target domain remote sensing image). Specifically, pixel data of the remote sensing image is read from the source domain to obtain the source domain labeled image, and the corresponding source domain label file is parsed to obtain the source domain bounding boxes and source domain category labels; pixel data of the unlabeled remote sensing image is read from the target domain to obtain the target domain unlabeled image. The source domain labeled image and the target domain unlabeled image serve together as input for subsequent multi-scale feature extraction, while the source domain bounding boxes and category labels also serve as input for subsequent prototype library updates and detection-supervised training.

[0050] S102 utilizes the shared feature extraction network of the target detection model to extract detection feature maps at multiple scales from source domain remote sensing images and target domain remote sensing images.

[0051] The shared feature extraction network refers to the part of the object detection model responsible for extracting basic visual features from the input image and unifying feature maps of different resolutions to a common channel dimension. This includes, but is not limited to, the backbone network and the multi-scale feature projection module. The backbone network is used to extract the basic features of the remote sensing image. The multi-scale feature projection module unifies the channel dimension of the feature maps of different resolutions output by the backbone network through convolutional or linear layers, providing a common feature source for subsequent frequency domain modulation and the main detection path.

[0052] Multi-scale detection feature maps refer to feature maps with different spatial resolutions output from different layers of a shared feature extraction network, used to cover targets to be detected at different scales. Specifically, for the first... The detection feature map can be denoted as a scale. Including source domain features and target domain features Among them, source domain features and target domain features They have the same number of channels.

[0053] Preferably, in some embodiments, the following steps can be used: utilizing a shared feature extraction network of the target detection model to extract detection feature maps at multiple scales from the source domain remote sensing image and the target domain remote sensing image: The first step is to perform the same scaling, normalization, and batch organization processing on both the source domain remote sensing images and the target domain remote sensing images to form a mixed training batch. The second step involves inputting the mixed training batch into the target detection model, and using a shared feature extraction network to extract detection feature maps at multiple scales from the source domain remote sensing images and the target domain remote sensing images.

[0054] S103, based on the detection feature maps at each scale, determine the original amplitude spectrum, original phase spectrum, and category composition embedding corresponding to each scale.

[0055] Among them, the original amplitude spectrum represents the energy magnitude of each frequency component in the spectrum, reflecting the texture, style, and appearance information of the feature map; the original phase spectrum represents the phase angle of each frequency component in the spectrum, mainly carrying the spatial structure, edge position, and geometric layout information of the feature map; the category composition embedding is a vector representing the probability of each category appearing in the current feature map, the dimension of which is equal to the total number of categories, used to adaptively adjust the modulation intensity according to the category distribution of the current feature during frequency domain modulation.

[0056] Specifically, in some embodiments, the original amplitude spectrum and original phase spectrum corresponding to each scale can be determined based on the detection feature maps at each scale through the following steps: The first step is to perform frequency domain transformation on the detection feature maps at each scale to obtain the frequency domain complex spectrum corresponding to each scale; For example, for the first Detection feature maps at various scales The frequency domain transform can be calculated using the following expression: ; In the formula, Represents the complex spectrum in the frequency domain; Represents the two-dimensional real number fast Fourier transform; Generally refers to the first Detection feature maps at various scales (applicable to both source and target domains); The second step is to perform amplitude-phase separation on the frequency domain complex spectrum corresponding to each scale to obtain the original amplitude spectrum and the original phase spectrum corresponding to each scale.

[0057] For example, amplitude-phase separation can be calculated using the following expression: ; ; In the formula, Represents the original amplitude spectrum; Represents the original phase spectrum; This indicates the amplitude value retrieval operation. This indicates the operation of taking the phase angle.

[0058] Additionally, it is necessary to note that the frequency domain transformation step does not alter the original detection feature map in the main detection path. .

[0059] Furthermore, in some embodiments, the embedding of the corresponding category at each scale can be determined based on the detection feature maps at each scale through the following steps: The first step is to extract the spatial features within the bounding box regions of each category at each scale from the detection feature maps at each scale.

[0060] For example, for the first The first scale, the first For each category, iterate through the source domain bounding boxes belonging to that category and extract the channel feature vectors corresponding to each spatial location within the bounding box. As an example, the extraction method can be, but is not limited to, direct spatial indexing (extracting feature vectors position by position), grid sampling (uniformly sampling a fixed number of position points within the bounding box and extracting the corresponding features), or dense extraction (extracting feature vectors for all spatial locations within the bounding box).

[0061] The second step is to perform spatial pooling aggregation on the spatial features within the corresponding bounding box area for each category at each scale, so as to obtain the temporary category prototypes corresponding to each category at each scale.

[0062] For example, for the first The first scale, the first For each category, the source domain bounding boxes can be mapped to the feature map coordinates of the corresponding scale. Then, the spatial location features within the source domain bounding box region belonging to the corresponding category are extracted. The temporary category prototype for the current batch is obtained by performing average pooling or RoIPool / RoIAlign-type region pooling on the spatial location features covered by the source domain bounding boxes on the feature map of the corresponding scale. .

[0063] The third step is to update the source domain feature prototype library for each category at each scale using a moving average method, based on the temporary category prototypes corresponding to each category at each scale.

[0064] Among them, the source domain feature prototype library refers to a data structure indexed by categories, storing representative vector representations of each category in the feature space of the source domain, used to estimate the category composition information of the current feature on unlabeled target domain features. For the first... The first scale, the first There are 1 category, and the source domain feature prototype is denoted as . Its dimension is the same as the number of channels in the detection feature map.

[0065] Preferably, in some embodiments, the source domain feature prototypes of each category at each scale are updated according to the following formula: ; In the formula, For the updated version The first scale, the first Source domain feature prototypes for each category, This is the prototype update coefficient, and its value ranges from 0.90 to 0.999. As an example, it can be set to 0.99. This is a temporary category prototype.

[0066] Additionally, it is worth noting that if the current batch does not contain the first... Class targets, then maintain No updates.

[0067] The fourth step is to determine the category composition embedding for each scale based on the detection feature maps at each scale and the updated source domain feature prototype library.

[0068] Preferably, the category composition embedding at each scale can be determined by the following steps based on the detection feature maps at each scale and the updated source domain feature prototype library: The first sub-step is to calculate the feature similarity between the features at each spatial location in the detection feature map and the source domain feature prototypes of each category in the source domain feature prototype library for each scale. As an example, the feature similarity is cosine similarity, and the feature similarity between the features at each spatial location and the source domain feature prototypes of each category is determined by the following formula: ; In the formula, For the first The first scale The spatial location and the first Feature similarity between source domain feature prototypes of each category For the first The first scale Characteristics of a spatial location For the first The first scale, the first The prototype of source domain features for each category.

[0069] The second sub-step involves spatial maximum aggregation of the non-negative responses of feature similarities for each category to obtain aggregated similarities for each category. The third sub-step involves normalizing the aggregated similarity of each category along the category dimension, and determining the normalized probability distribution vector as the category composition embedding corresponding to each scale.

[0070] As an example, the category composition embedding corresponding to each scale is determined by the following formula: ; In the formula, For the first The categories corresponding to each scale form the embedding. For feature similarity, Total number of categories; Indicates spatial location index, This indicates taking the maximum value across all spatial locations. This represents a modified linear unit, whose function is to set the negative similarity response to zero and retain only the non-negative response; This indicates normalization along the category dimension, for example, it can be done using... Normalization is performed so that the sum of the normalized values ​​for each category is 1. The purpose of the above steps is to convert the unlabeled target domain features into a category composition description that can be used for conditional modulation.

[0071] S104: Based on the learnable scale embedding parameters and categories corresponding to each scale, an embedding is formed to generate the channel modulation factor corresponding to each scale.

[0072] Here, the learnable scale embedding parameter refers to a vector parameter set for each feature scale that can be updated through gradient descent. It is used to indicate whether the current feature comes from a low-level texture scale, a mid-level structural scale, or a high-level semantic scale, and is denoted as . Its dimensions are The channel modulation factor is a vector with one channel dimension, whose elements control the degree of enhancement or suppression of the low-frequency amplitude spectrum of the corresponding channel, denoted as . .

[0073] Specifically, in some embodiments, the channel modulation factors corresponding to each scale can be generated through the following steps: The first step is to combine the learnable scale embedding parameters and categories corresponding to each scale into an embedding fusion vector of scale categories. As an example, categories can be composed of embeddings. With learnable scale embedding parameters The scale category condition vector is concatenated to obtain a scale category condition vector that simultaneously contains two types of information: "which scale is currently being used" and "which categories the current feature is closer to." Specifically, this can be determined using the following formula: ; In the formula, This is the scale category condition vector.

[0074] The second step is to input the scale category condition vector into the condition vector mapping network to obtain the mapping output; The third step involves applying an activation function to the mapped output to constrain the numerical range, thereby obtaining the channel modulation factor corresponding to each scale. As an example, the channel modulation factor corresponding to each scale is determined by the following formula:

[0075] ; In the formula, For the first The channel modulation factor corresponding to each scale This is a scale-category condition vector. It is a conditional vector mapping network, specifically a multilayer perceptron, which may contain one or more fully connected layers, normalized layers, and nonlinear activation layers; The activation function is specifically the Sigmoid function, whose output ranges from 0 to 1. Multiplying it by a coefficient of 2 makes... The value ranges from 0 to 2, with a center value of 1.

[0076] Among them, when When the value of a certain channel is greater than 1, it indicates that the low-frequency amplitude of the corresponding channel is enhanced; when it is less than 1, it indicates that the low-frequency amplitude of the corresponding channel is suppressed; when it is equal to 1, it indicates that the low-frequency amplitude of the corresponding channel remains unchanged.

[0077] S105, based on the channel modulation factor corresponding to each scale, modulates the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale to obtain the modulated amplitude spectrum corresponding to each scale.

[0078] Among them, the amplitude value of the low-frequency region refers to the amplitude value corresponding to the frequency component close to the DC component (i.e., the zero frequency position) in the amplitude spectrum. These components mainly carry the domain difference information such as the global appearance, brightness and color style of the image or feature; the modulated amplitude spectrum refers to the amplitude spectrum obtained by applying the channel modulation factor to the amplitude value of the low-frequency region on the basis of the original amplitude spectrum.

[0079] Specifically, in some embodiments, the step of modulating the amplitude values ​​of the low-frequency region in the original amplitude spectrum corresponding to each scale to obtain the modulated amplitude spectrum corresponding to each scale includes: The first step is to construct a low-frequency smoothing mask for each scale based on the frequency domain size of the original amplitude spectrum. The low-frequency smoothing mask takes a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region. As an example, the low-frequency smoothing mask is determined by the following formula: ; In the formula, For the first Each scale at frequency position The low-frequency smoothing mask value is close to 1 at low-frequency locations and close to 0 at high-frequency locations. This is the normalized distance from the frequency position to the DC component position. For the first A low-frequency bandwidth parameter at each scale is used to control the decay rate of the low-frequency smoothing mask from low frequency to high frequency.

[0080] If the frequency distance uses a normalized distance, then we can let If the frequency distance uses the pixel index distance, then it can be set as follows: In this formula, For low-frequency ratios, the value can range from 0.10 to 0.20, for example, it can be set to 0.15; The frequency domain feature size.

[0081] The second step is to broadcast the channel modulation factor corresponding to each scale to the same shape as the original amplitude spectrum, and control the modulation region according to the low-frequency smoothing mask to obtain the modulated amplitude spectrum corresponding to each scale.

[0082] As an example, the modulated amplitude spectrum corresponding to each scale is determined by the following formula: ; In the formula, For the first The modulated amplitude spectrum corresponding to each scale. The original amplitude spectrum, For low-frequency smoothing masks, For channel modulation factor, This represents element-wise multiplication. When... When the frequency is close to 1, the current frequency position is mainly affected by Modulation; when When the frequency is close to 0, the current frequency position basically maintains the original amplitude. constant.

[0083] S106. Based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale, perform inverse frequency domain transformation to obtain the modulated spatial features corresponding to each scale.

[0084] Specifically, in some embodiments, the step of performing inverse frequency domain transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale to obtain the modulated spatial features corresponding to each scale includes: The first step is to reconstruct the frequency domain complex spectrum based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale. The second step is to perform an inverse frequency domain transformation on the reconstructed frequency domain complex spectrum to obtain the modulated spatial features corresponding to each scale.

[0085] As an example, the modulated spatial features corresponding to each scale are determined by the following formula: ; In the formula, For the first Modulated spatial features corresponding to each scale The amplitude spectrum after modulation. The original phase spectrum, The imaginary unit satisfies , This is the inverse two-dimensional real-number fast Fourier transform. Because... The original phase spectrum is always derived from S103. This step only changes the low-frequency amplitude-related information and does not directly change the phase structure.

[0086] S107, based on the modulated spatial features corresponding to each scale, optimize the learnable scale embedding parameters and the parameters of the target detection model.

[0087] Specifically, in some embodiments, the learnable scale embedding parameters and the parameters of the target detection model can be optimized based on the modulated spatial features corresponding to each scale through the following steps: The first step is to fuse the modulated spatial features corresponding to each scale with the detection feature maps corresponding to each scale using residual reconstruction to obtain the training-period adversarial features corresponding to each scale. As an example, the training-period adversarial features corresponding to each scale are determined by the following formula:

[0088] ; In the formula, For the first The training-period adversarial features corresponding to each scale; To detect feature maps; The modulated spatial features; The weights are reconstructed from the learnable residuals. The initial value can be set to 0, and they are optimized as learnable parameters during training through gradient descent, so that the identity mapping is maintained in the early stage of training to ensure training stability.

[0089] The second step involves inputting the adversarial features from the training period corresponding to each scale into the domain discriminator, and applying domain confusion constraints to the backbone network and feature projection module of the shared feature extraction network through a gradient inversion layer to obtain the domain adversarial loss corresponding to each scale.

[0090] The domain discriminator determines whether the input features originate from the source domain or the target domain. It may include convolutional layers, linear layers, normalization layers, and a classification output layer. For example, it can classify the adversarial features from the source and target domains during training. The input domain discriminator is used, and the gradient reversal layer multiplies the gradients propagated to the backbone network and feature projection module of the shared feature extraction network by a negative coefficient during backpropagation. This makes the shared feature extraction network learn the low-frequency amplitude representations that are more difficult to distinguish between the source and target domains, thus obtaining the domain adversarial loss at various scales. .

[0091] The third step is to determine the detection supervision loss based on the bounding boxes and category labels of the source domain remote sensing image.

[0092] Specifically, during training, the main detection path of the object detection model uses the detection feature map directly provided by the shared feature extraction network as input, that is, it uses the original multi-scale detection feature map output by S102. (Instead of modulated spatial features), the main detection path of the aforementioned target detection model (including a detection encoder, feature fusion module, detection decoder, and detection head, where the detection head may include a classification prediction head, a bounding box regression head, and an overlap prediction head) is input. Then, for source domain samples, the classification loss, bounding box regression loss, and overlap loss are calculated based on the labeled boxes and class labels provided in S101; next, the detection supervision loss is calculated as a weighted sum of the classification loss, bounding box regression loss, and overlap loss to obtain the detection supervision loss. Specifically, the original multi-scale detection feature map output by S102 is used. (Instead of modulated spatial features), inputting the main detection path of the aforementioned target detection model can prevent the main detection path features from being replaced, thereby avoiding the modulation features from damaging the spatial localization capability of the detector.

[0093] The fourth step involves iteratively updating the learnable scale embedding parameters and the parameters of the object detection model using the weighted sum of the detection supervision loss and the domain adversarial loss corresponding to each scale as the total loss, until the total loss converges or the preset number of training rounds is reached.

[0094] As an example, the total loss is determined by the following formula: ; In the formula, Total loss; To detect and monitor losses; For the first Domain adversarial loss corresponding to each scale; Loss weights for domain adversarial operations; This represents the total number of scales, for example, it can be set to 3 to 5.

[0095] For example, in each training iteration, based on the total loss Calculate the gradient and use the optimizer to perform parameter updates. For example, the optimizer could be stochastic gradient descent, Adam, or AdamW.

[0096] For ease of description, the following text will use the weighted sum of the detection supervision loss and the domain adversarial loss corresponding to each scale as the total loss, and the parameters updated iteratively (weights, biases, normalized layer affine parameters and embedding parameters, and learnable residual reconstruction weights that can be updated through gradient descent in each functional module) as the parameters updated iteratively. ), denoted as the set of learnable parameters. .

[0097] ; Specifically, the backbone network parameters of the shared feature extraction network Learnable parameters, including affine parameters of convolutional layers, Transformer layers, and normalized layers, used to extract fundamental features of remote sensing images; parameters of the multi-scale feature projection module. This includes parameters for convolutional layers, linear layers, or normalized layers used to unify the number of feature channels across multiple scales; and parameters for the main detection path. This includes learnable parameters in the detection encoder, feature fusion module, detection decoder, classification prediction head, bounding box regression head, and overlap prediction head; where the detection encoder and detection decoder include, but are not limited to, corresponding functional modules in models with explicit encoder-decoder structures (such as DETR-type detection models), and corresponding feature encoding and decoding functional modules in single-stage detection models (such as the YOLO series); and parameters of the scale-category conditional modulation module. This includes learnable scale embedding parameters and conditional vector mapping networks; domain discriminator parameters. This includes parameters for convolutional layers, linear layers, normalized layers, and classification output layers in a domain classifier; and learnable residual reconstruction weights. It is part of the S107 residual fusion calculation.

[0098] Detect the main path parameters along the gradient flow direction. It primarily receives the gradient of the detection supervision loss; it shares the backbone network parameters of the feature extraction network and the parameters of the feature projection module. Simultaneously receive the detection supervision loss gradient and the domain confusion gradient propagated in reverse by the gradient inversion layer; scale-class conditional modulation module parameters. Domain adversarial loss gradient in the received frequency domain modulation branch; domain discriminator parameters The normal gradient of the adversarial loss in the receiver domain. It should be noted that the parameters of the domain discriminator... Losses due to domain confrontation Independent optimization is performed, decoupled from the joint optimization of the shared feature extraction network and the scale-category conditional modulation module. Parameter updates can be calculated using the following expression:

[0099] ; in, This represents the training parameters obtained after joint optimization. Indicates the optimizer; Indicates total loss Regarding the set of learnable parameters The gradient.

[0100] The gradient inversion layer itself does not contain learnable parameters; its function is to change the gradient direction propagating to the shared feature extraction network during backpropagation. The source domain feature prototype library is maintained using an exponential moving average and is not a learnable parameter updated via gradient descent. The low-frequency smoothing mask is calculated from the frequency domain size and low-frequency ratio and is not a learnable parameter. Frequency domain transform, inverse frequency domain transform, and the gradient inversion layer also do not contain learnable parameters. (Low-frequency ratio...) Prototype update coefficient Domain adversarial loss weights These are preset hyperparameters or training scheduling parameters.

[0101] The above training process continues iteratively until the total loss converges or the preset number of training rounds is reached. During the model inference phase, only the shared feature extraction network and detection main path of S102 are retained. Frequency domain transformation, frequency domain modulation, inverse frequency domain transformation, domain discriminator, and gradient inversion layer are not enabled, thus not increasing the computational overhead of frequency domain transformation, inverse frequency domain transformation, or domain discriminator.

[0102] In summary, based on Figure 1 The training method for a remote sensing cross-domain target detection model shown in this application has at least the following technical effects: 1. By determining the original amplitude spectrum, original phase spectrum, and category composition embedding based on the detection feature map, the amplitude information, phase information, and category semantic information of the detection feature map can be separated. In this way, a channel modulation factor can be generated jointly by the learnable scale embedding parameters and the category composition embedding. Applying the channel modulation factor to the low-frequency region of the original amplitude spectrum allows the modulation intensity to simultaneously match the current scale level and the category composition in the current feature, while the modulation operation is confined to the low-frequency amplitude component range.

[0103] Building upon this, by performing an inverse frequency domain transformation between the modulated amplitude spectrum and the original phase spectrum to obtain the modulated spatial features, the original phase spectrum remains numerically unchanged throughout the entire process from determination to inverse transformation. Therefore, the modulated spatial features possess the characteristic of adaptively adjusting the amplitude distribution while fully preserving spatial structure information. Consequently, when the modulated spatial features are used to simultaneously optimize the learnable scale embedding parameters and the parameters of the target detection model, allowing the modulation strategy to converge collaboratively with the detection task objective during training, the target detection model can automatically adapt to the distribution differences between the source and target domains. This ensures that the trained target detection model eliminates the appearance differences between the source and target domains without damaging the target's geometry, thereby improving the accuracy of cross-domain target detection in remote sensing.

[0104] 2. By spatially pooling and aggregating the spatial features within the bounding boxes of various categories at different scales, temporary category prototypes for each category are obtained. These temporary category prototypes are then fused into the source domain feature prototype library using a moving average method. This allows the source domain feature prototype library to gradually accumulate stable feature representations of each category at each scale during training. Specifically, spatial pooling can compress features from multiple spatial locations within the bounding box into a single prototype vector, eliminating the interference of individual abnormal pixels within the box on prototype estimation. Moving average updates allow new batches of feature information to be integrated into the prototype library at a controlled rate, avoiding drastic fluctuations in prototypes caused by accidental noise in a single batch of samples. Thus, the source domain feature prototype library can provide a stable and reliable reference basis for subsequent category composition estimation, ensuring that the category composition embedding calculated based on the prototype library accurately reflects the category distribution of the current detection feature map, thereby providing a reliable category condition input for the channel modulation factor.

[0105] 3. By calculating the feature similarity between features at each spatial location in the detection feature map and source domain feature prototypes of each category in the source domain feature prototype library, and then performing spatial maximum aggregation after taking the non-negative response of the feature similarity for each category, followed by normalization along the category dimension, it is possible to generate category composition embeddings reflecting the presence intensity of each category of target in the detection feature map at each scale without relying on manual annotation of the target domain. Specifically, spatial maximum aggregation can use the most significant response position of each category in the entire detection feature map as the criterion for the presence or absence of that category, thus avoiding the need for explicit detection of the target spatial range; non-negative response filtering can eliminate the interference of irrelevant feature vectors with directions opposite to those of the source domain feature prototypes; normalization can ensure that the category composition embeddings of different scales and batches maintain a consistent numerical scale. Thus, meaningful category composition embeddings can also be obtained for remote sensing images of the target domain, allowing the generation of channel modulation factors to be differentiated according to the implicit category distribution in the detection feature map.

[0106] 4. By constructing a low-frequency smoothing mask based on the frequency domain dimensions of the original amplitude spectrum, and ensuring that the low-frequency smoothing mask has a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region, a continuous modulation weight transition can be formed between the low-frequency and high-frequency regions. Based on this, the channel modulation factors are broadcast to the same shape as the original amplitude spectrum, and the modulation region is controlled according to the low-frequency smoothing mask. This ensures that the change in modulation intensity from low frequency to high frequency is smooth and gradual, rather than abruptly occurring at a certain frequency position. Therefore, it avoids the ringing effect or characteristic instability caused by introducing abrupt boundary changes in the frequency domain, ensuring that the spectral quality of the modulated amplitude spectrum is not affected by artifacts.

[0107] 5. By fusing the modulated spatial features with the detection feature map using residual reconstruction, adversarial features are obtained during the training phase. These adversarial features are then input into the domain discriminator and subjected to domain confusion constraints through a gradient inversion layer. This allows the frequency domain modulation branch to indirectly optimize the parameters of the shared feature extraction network only during the training phase through adversarial gradients, without directly replacing the input features of the main detection path. Simultaneously, the main detection path uses the detection feature map directly provided by the shared feature extraction network as input, ensuring that the spatial localization capability of the detection branch is not directly affected by the modulation features. This maintains the stability of the detection feature distribution and avoids interference with the spatial localization accuracy of target detection during the frequency domain alignment process.

[0108] 6. By separating the amplitude and phase spectra through frequency domain transformation, selectively modulating only low-frequency amplitudes, using scale embedding and category-based embedding to jointly determine the modulation intensity, and decoupling the modulated spatial features from the main detection path and using them only for the adversarial branch during training, the differences in low-frequency energy distribution between source and target domain detection features can be gradually reduced during training, while protecting the spatial structure and high-frequency detail information carried by the phase spectrum. After the source and target domain detection features become more consistent in their low-frequency amplitude distribution, the features extracted by the target detection model under different sensor platforms, spatial resolutions, and imaging conditions are more consistent. This results in higher detection stability, lower false negative rate, and lower background false positive rate in the target domain, enhancing the cross-domain adaptability of the target detection model under conditions without target domain annotation.

[0109] To facilitate understanding, the application scenarios and parameter configurations of the above methods will be further explained below with reference to specific embodiments.

[0110] Example 1: Cross-satellite remote sensing target detection training Taking cross-satellite remote sensing target detection as an example, the source domain is a labeled remote sensing image of satellite platform A, and the target domain is an unlabeled remote sensing image of satellite platform B. The source and target domains share the same detection category, such as aircraft, ships, vehicles, or storage tanks. Before training, the labeled images in the source domain and the unlabeled images in the target domain are unified to the input size required by the target detection model, and the same target detection model is used to extract multi-scale detection feature maps.

[0111] During training, each batch can contain a certain number of source domain labeled images and target domain unlabeled images. Source domain labeled images are used to calculate the detection supervision loss and to update the source domain feature prototype library; target domain unlabeled images do not participate in the calculation of the detection supervision loss, but participate in frequency domain selective alignment. For the ... At each scale, the corresponding source domain feature prototype is updated based on the features within the source domain bounding box. Based on the aforementioned prototype, the category composition embedding of the current source or target domain features is estimated. .

[0112] Subsequently, the learnable scale embedding parameters and category composition embeddings are fused into a scale-category conditional vector. The channel modulation factor is obtained through a conditional vector mapping network. After performing frequency domain transformation and amplitude-phase separation on the above-mentioned scale detection feature map, the original amplitude spectrum is obtained. and the original phase spectrum Further based on the proportion of low frequencies Constructing a low-frequency smoothing mask Only the low-frequency amplitude is modulated, and the original phase spectrum is used to perform inverse frequency domain transformation reconstruction. The reconstructed training-period adversarial features are fed into the domain discriminator; the main detection path still uses the original detection feature map to ensure decoupling between the detection branch and the frequency domain alignment branch.

[0113] Example 2: Satellite to UAV Scene Migration In satellite-to-UAV scene migration, source domain labeled images typically have a large coverage area and a relatively stable top-down view, while target domain UAV remote sensing images may exhibit more significant variations in shooting altitude, viewpoint, and local texture differences. For small targets such as vehicles and ships, the low-frequency amplitude distribution in the target domain may differ significantly from that in the source domain, while the target's outline, center position, and spatial geometry still need to remain stable. In this embodiment, the training method of the aforementioned remote sensing cross-domain target detection model is used to perform frequency-domain selective modulation on detection feature maps at three or more scales. Lower-scale features with a higher concentration of small targets can obtain stronger category response modulation, while features at higher semantic scales can obtain smoother domain appearance correction.

[0114] Parameter examples In a feasible configuration, the number of feature scales It can be set to 3, the prototype update coefficient. It can be set to 0.99, low frequency ratio It can be set to 0.15 to learn the residual reconstruction weights. The initial value can be set to 0. Channel modulation factor. The output can be from a conditional vector mapping network (e.g., a multilayer perceptron) and is limited to the range of 0 to 2. It is important to note that the above parameters are merely illustrative embodiments and do not constitute a limitation on the scope of protection. The parameters can be adjusted according to the degree of spectral difference under different sensors, different ground sampling distances, or different target categories. , Scale number and domain adversarial loss weights.

[0115] Based on the same inventive concept as the training method of a remote sensing cross-domain target detection model in the foregoing embodiments, please refer to... Figure 2 This application provides a training device for a remote sensing cross-domain target detection model, comprising: The data acquisition module 201 is used to acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images; The multi-scale feature extraction module 202 is used to extract detection feature maps at multiple scales from source domain remote sensing images and target domain remote sensing images by utilizing the shared feature extraction network of the target detection model. The first generation module 203 is used to determine the original amplitude spectrum and original phase spectrum corresponding to each scale, as well as the category composition embedding, based on the detection feature maps at each scale. The second generation module 204 is used to generate the channel modulation factor corresponding to each scale by forming an embedding based on the learnable scale embedding parameters and category corresponding to each scale. The low-frequency amplitude modulation module 205 is used to modulate the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale according to the channel modulation factor corresponding to each scale, so as to obtain the modulated amplitude spectrum corresponding to each scale. The spatial feature reconstruction module 206 is used to perform frequency domain inverse transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale to obtain the modulated spatial features corresponding to each scale. The parameter optimization module 207 is used to optimize the learnable scale embedding parameters and the parameters of the target detection model based on the modulated spatial features corresponding to each scale.

[0116] In some embodiments, the first generation module 203 is specifically used for: A frequency domain transformation is performed on the detection feature maps at each scale to obtain the frequency domain complex spectrum corresponding to each scale; Amplitude and phase separation is performed on the frequency domain complex spectrum corresponding to each scale to obtain the original amplitude spectrum and the original phase spectrum corresponding to each scale.

[0117] In some embodiments, the annotation of the source domain remote sensing image includes annotation boxes, and the first generation module 203 is further configured to: From the detection feature maps at various scales, extract the spatial features within the bounding box regions of each category at each scale; For each category at each scale, spatial pooling is performed on the spatial features within the corresponding bounding box area to obtain temporary category prototypes for each category at each scale. Based on the temporary category prototypes corresponding to each category at each scale, the source domain feature prototype library for each category at each scale is updated by moving average. Based on the detection feature maps at each scale and the updated source domain feature prototype library, the category composition embedding at each scale is determined.

[0118] In some embodiments, the source domain feature prototype library for each scale and category is updated using the following formula: ; In the formula, For the updated version The first scale, the first The prototype of source domain features for each category, For prototype update coefficients, This is a temporary category prototype.

[0119] In some embodiments, the first generation module 203 is further configured to: For each scale, calculate the feature similarity between the features at each spatial location in the detected feature map and the source domain feature prototypes of each category in the source domain feature prototype library; Spatial maximum aggregation is performed on the non-negative responses of feature similarity for each category to obtain aggregated similarity for each category; The aggregated similarity of each category is normalized along the category dimension, and the normalized probability distribution vector is determined as the category composition embedding corresponding to each scale.

[0120] In some embodiments, the feature similarity is cosine similarity, and the feature similarity between the features at each spatial location and the source domain feature prototypes of each category is determined by the following formula: ; In the formula, For the first The first scale The spatial location and the first Feature similarity between source domain feature prototypes of each category For the first The first scale Characteristics of a spatial location For the first The first scale, the first The prototype of source domain features for each category.

[0121] In some embodiments, the category composition embedding corresponding to each scale is determined by the following formula: ; In the formula, For the first The categories corresponding to each scale form the embedding. For feature similarity, Total number of categories; Indicates spatial location index, This indicates taking the maximum value across all spatial locations. Indicates a corrected linear unit. This indicates normalization along the category dimension.

[0122] In some embodiments, the second generation module 204 is specifically used for: For each scale, the learnable scale embedding parameters and categories corresponding to the scale are combined into an embedding fusion vector to form a scale category condition vector; Input the scale category condition vector into the condition vector mapping network to obtain the mapping output; An activation function is applied to the mapped output to constrain the numerical range, thus obtaining the channel modulation factor corresponding to each scale.

[0123] In some embodiments, the channel modulation factor corresponding to each scale is determined by the following formula: ; In the formula, For the first The channel modulation factor corresponding to each scale This is a scale-category condition vector. For conditional vector mapping networks, This is the activation function.

[0124] In some embodiments, the low-frequency amplitude modulation module 205 is specifically used for: For each scale, a low-frequency smoothing mask is constructed based on the frequency domain size of the original amplitude spectrum. The low-frequency smoothing mask takes a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region. For each scale, the channel modulation factor corresponding to the scale is broadcast to the same shape as the original amplitude spectrum, and the modulation region is controlled according to the low-frequency smoothing mask to obtain the modulated amplitude spectrum corresponding to each scale.

[0125] In some embodiments, the low-frequency smoothing mask is determined by the following formula: ; In the formula, For the first Each scale at frequency position Low-frequency smoothing mask value at the location, This is the normalized distance from the frequency position to the DC component position. For the first Low-frequency bandwidth parameters at each scale.

[0126] In some embodiments, the modulated amplitude spectrum corresponding to each scale is determined by the following formula: ; In the formula, For the first The modulated amplitude spectrum corresponding to each scale. The original amplitude spectrum, For low-frequency smoothing masks, For channel modulation factor, This indicates element-wise multiplication.

[0127] In some embodiments, the spatial feature reconstruction module 206 is specifically used for: For each scale, the frequency domain complex spectrum is reconstructed based on the modulated amplitude spectrum and the original phase spectrum corresponding to the scale. The reconstructed frequency domain complex spectrum is subjected to inverse frequency domain transformation to obtain the modulated spatial features corresponding to each scale.

[0128] In some embodiments, the modulated spatial features corresponding to each scale are determined by the following formula: ; In the formula, For the first Modulated spatial features corresponding to each scale The amplitude spectrum after modulation. The original phase spectrum, The imaginary unit, It is the inverse two-dimensional real fast Fourier transform.

[0129] In some embodiments, the parameter optimization module 207 is specifically used for: The modulated spatial features corresponding to each scale are fused with the detection feature maps corresponding to each scale using a residual reconstruction method to obtain the training-period adversarial features corresponding to each scale. The adversarial features corresponding to each scale during the training period are input into the domain discriminator, and the domain confusion constraint is applied to the backbone network and feature projection module of the shared feature extraction network through the gradient inversion layer to obtain the domain adversarial loss corresponding to each scale. Based on the bounding boxes and category labels of the source domain remote sensing images, determine the detection supervision loss; The total loss is the weighted sum of the detection supervision loss and the domain adversarial loss corresponding to each scale. The learnable scale embedding parameters and the parameters of the object detection model are iteratively updated until the total loss converges or the preset training rounds are reached. During the training process, the main detection path of the object detection model takes the detection feature map directly provided by the shared feature extraction network as input.

[0130] In some embodiments, the training-period adversarial features corresponding to each scale are determined by the following formula: ; In the formula, For the first The training-period adversarial features corresponding to each scale To detect feature maps, For modulated spatial features, Weights are reconstructed from learnable residuals.

[0131] In some embodiments, the total loss is determined by the following formula: ; In the formula, For the total loss, In order to detect and monitor losses, For the first Domain adversarial loss corresponding to each scale For domain adversarial loss weights, This represents the total number of scales.

[0132] It is understandable that the modules recorded in the training device of this remote sensing cross-domain target detection model are similar to... Figure 1The steps in the training method for the remote sensing cross-domain target detection model described above correspond to each other. Therefore, the operations, features, and beneficial effects described above also apply to the training device and its modules for the remote sensing cross-domain target detection model, and will not be repeated here.

[0133] The modules described in some embodiments of this application can be implemented in software or hardware. The described modules can also be housed in a processing device. Furthermore, the names of these modules do not necessarily limit the module itself; for example, the data acquisition module 201 can also be described as a "data receiving module".

[0134] In particular, embodiments of this application also provide a cross-domain remote sensing image detection method, including: Acquire the remote sensing image to be detected; The shared feature extraction network of the remote sensing cross-domain target detection model is used to extract detection feature maps of multiple scales from the remote sensing image to be detected; wherein, the remote sensing cross-domain target detection model is trained by the training method of the remote sensing cross-domain target detection model described in any of the above embodiments; The detection feature maps at each scale are input into the main detection path of the remote sensing cross-domain target detection model to obtain the target detection results at each scale.

[0135] In some embodiments, the remote sensing cross-domain target detection model retains only the shared feature extraction network and detection main path from S102 during the inference phase, without enabling frequency domain transformation, frequency domain modulation, inverse frequency domain transformation, domain discriminator, and gradient inversion layer. With this configuration, once the model enters the inference phase, frequency domain transformation, frequency domain modulation, inverse frequency domain transformation, domain discriminator, and gradient inversion layer are no longer enabled. Therefore, FFT calculation, inverse FFT calculation, and domain discriminator discrimination are unnecessary, resulting in no additional computational overhead during deployment and no increase in model structural complexity.

[0136] Please see Figure 3Based on the same inventive concept as the training method for a cross-domain remote sensing target detection model or a cross-domain remote sensing image detection method in the foregoing embodiments, this application provides an electronic device. This electronic device may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device includes a processing unit 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM 302 (read-only memory) or a program loaded from storage device 308 into RAM 303 (random access memory). RAM 303 also stores various programs and data required for the operation of the electronic device. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output interface (i.e., I / O interface 305) is also connected to the bus 304.

[0137] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data.

[0138] In particular, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is implemented when executed by the processing device 301. Figure 1 The training method of the remote sensing cross-domain target detection model shown or the aforementioned cross-domain remote sensing image detection method.

[0139] The computer-readable medium described in some embodiments of this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CDROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0140] In some embodiments of this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In some embodiments of this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0141] Additionally, it is worth noting that the aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently without being assembled into the electronic device.

[0142] Finally, it is necessary to note that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of the systems and methods according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A training method for a remote sensing cross-domain target detection model, characterized in that, Includes the following steps: Acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images; Using the shared feature extraction network of the target detection model, detection feature maps of multiple scales are extracted from the source domain remote sensing image and the target domain remote sensing image; Based on the detection feature maps of each scale, determine the original amplitude spectrum, original phase spectrum, and category composition embedding corresponding to each scale; Based on the learnable scale embedding parameters corresponding to each scale and the category, an embedding is formed to generate the channel modulation factor corresponding to each scale; Based on the channel modulation factor corresponding to each scale, the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale is modulated to obtain the modulated amplitude spectrum corresponding to each scale. Based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale, the frequency domain inverse transform is performed to obtain the modulated spatial features corresponding to each scale. Based on the modulated spatial features corresponding to each scale, optimize the learnable scale embedding parameters and the parameters of the target detection model.

2. The training method for the remote sensing cross-domain target detection model according to claim 1, characterized in that, The annotation of the source domain remote sensing image includes a bounding box, and the step of determining the category composition embedding corresponding to each scale based on the detection feature map of each scale includes: From the detection feature maps of each scale, extract the spatial features within the bounding box area of ​​each category at each scale; For each category at each scale, the spatial features within the corresponding bounding box area are spatially pooled and aggregated to obtain temporary category prototypes for each category at each scale. Based on the temporary category prototypes corresponding to each category at each scale, the source domain feature prototype library for each category at each scale is updated by a moving average method. Based on the detection feature maps of each scale and the updated source domain feature prototype library, the category composition embedding of each scale is determined.

3. The training method for the remote sensing cross-domain target detection model according to claim 2, characterized in that, The step of determining the category composition embedding for each scale based on the detected feature maps at each scale and the updated source domain feature prototype library includes: For each scale, calculate the feature similarity between the features at each spatial location in the detection feature map and the source domain feature prototypes of each category in the source domain feature prototype library; Spatial maximum aggregation is performed on the non-negative responses of the feature similarities of each category to obtain the aggregated similarities of each category; The aggregated similarity of each category is normalized along the category dimension, and the normalized probability distribution vector is determined as the category composition embedding corresponding to each scale.

4. The training method for the remote sensing cross-domain target detection model according to claim 1, characterized in that, The step of generating channel modulation factors corresponding to each scale by forming an embedding based on the learnable scale embedding parameters corresponding to each scale and the category includes: For each scale, the learnable scale embedding parameters corresponding to the scale and the category are combined to form an embedding fusion into a scale category condition vector; The scale category condition vector is input into the condition vector mapping network to obtain the mapping output; An activation function is applied to the mapping output to constrain the numerical range, thereby obtaining the channel modulation factor corresponding to each scale.

5. The training method for the remote sensing cross-domain target detection model according to claim 1, characterized in that, The step of modulating the amplitude values ​​of the low-frequency region in the original amplitude spectrum corresponding to each scale according to the channel modulation factor corresponding to each scale, to obtain the modulated amplitude spectrum corresponding to each scale, includes: For each of the aforementioned scales, a low-frequency smoothing mask is constructed based on the frequency domain size of the original amplitude spectrum. The low-frequency smoothing mask has a value close to 1 in the low-frequency region and smoothly decays to close to 0 in the high-frequency region. For each scale, the channel modulation factor corresponding to the scale is broadcast to the same shape as the original amplitude spectrum, and the modulation region is controlled according to the low-frequency smoothing mask to obtain the modulated amplitude spectrum corresponding to each scale.

6. The training method for the remote sensing cross-domain target detection model according to claim 1, characterized in that, The step of optimizing the learnable scale embedding parameters and the parameters of the target detection model based on the modulated spatial features corresponding to each scale includes: The modulated spatial features corresponding to each scale and the detection feature map corresponding to each scale are fused by residual reconstruction to obtain the training period adversarial features corresponding to each scale. The training-period adversarial features corresponding to each scale are input into the domain discriminator, and domain confusion constraints are applied to the backbone network and feature projection module of the shared feature extraction network through the gradient inversion layer to obtain the domain adversarial loss corresponding to each scale. Based on the bounding boxes and category labels of the source domain remote sensing images, determine the detection supervision loss; The total loss is calculated by weighting the detection supervision loss and the domain adversarial loss corresponding to each scale. The learnable scale embedding parameters and the parameters of the target detection model are iteratively updated until the total loss converges or a preset number of training rounds are reached. During training, the main detection path of the target detection model takes the detection feature map directly provided by the shared feature extraction network as input.

7. A training device for a remote sensing cross-domain target detection model, characterized in that, include: The data acquisition module is used to acquire labeled source domain remote sensing images and unlabeled target domain remote sensing images; A multi-scale feature extraction module is used to extract detection feature maps of multiple scales from the source domain remote sensing image and the target domain remote sensing image by utilizing the shared feature extraction network of the target detection model. The first generation module is used to determine the original amplitude spectrum and original phase spectrum corresponding to each scale, as well as the category composition embedding, based on the detection feature maps of each scale. The second generation module is used to generate channel modulation factors corresponding to each scale based on the learnable scale embedding parameters corresponding to each scale and the category composition embedding; The low-frequency amplitude modulation module is used to modulate the amplitude value of the low-frequency region in the original amplitude spectrum corresponding to each scale according to the channel modulation factor corresponding to each scale, so as to obtain the modulated amplitude spectrum corresponding to each scale. The spatial feature reconstruction module is used to perform inverse frequency domain transformation based on the modulated amplitude spectrum and the original phase spectrum corresponding to each scale to obtain the modulated spatial features corresponding to each scale. The parameter optimization module is used to optimize the learnable scale embedding parameters and the parameters of the target detection model based on the modulated spatial features corresponding to each scale.

8. A cross-domain remote sensing image detection method, characterized in that, Includes the following steps: Acquire the remote sensing image to be detected; The remote sensing cross-domain target detection model uses a shared feature extraction network to extract detection feature maps of multiple scales from the remote sensing image to be detected; wherein the remote sensing cross-domain target detection model is trained by the training method of the remote sensing cross-domain target detection model as described in any one of claims 1 to 6; The detection feature maps at each scale are input into the main detection path of the remote sensing cross-domain target detection model to obtain the target detection results at each scale.

9. An electronic device, characterized in that, include: One or more processing devices; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processing devices, the one or more processing devices implement the method as claimed in claim 8 or any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processing device, implements the method of claim 8 or any one of claims 1 to 6.

Images