Domain generalization target detection method based on large model guidance and related equipment

Abstract

The embodiment of the application discloses a domain generalization target detection method based on a large model guide and related equipment, a training image set with the same content is obtained, and a foreground enhanced image set is generated in combination with foreground mask guidance, which not only helps to enrich the diversity of data enhancement from the root by means of multiple training data, but also focuses on the target foreground for feature enhancement through the foreground mask, thereby effectively retaining the positioning characteristics of the target while improving the richness of data distribution and avoiding damage to the positioning accuracy of the target. The style of the foreground enhanced image set and the to-be-predicted image is fused to realize feature adaptation, and then a hyper-domain invariant feature encoding is used to obtain a feature map after feature encoding. On the basis of retaining key feature information, the feature map after feature encoding efficiently extracts domain invariant features with strong generalization, thereby avoiding feature information loss and improving feature representation capability, so that accurate target detection results are obtained.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to a domain generalization target detection method and related equipment based on a large model-guided approach. Background Technology

[0002] Among existing single-source domain generalized target detection technologies, data augmentation-based single-source domain generalized target detection methods suffer from insufficient augmentation diversity and compromised target localization accuracy, while methods based on extracting domain-invariant features suffer from information loss and reduced feature representation capabilities. Both types of methods are limited by the inherent bottleneck of the single-source domain data distribution. Summary of the Invention

[0003] In view of this, the present invention provides a domain generalization target detection method and related equipment based on a large model.

[0004] The specific technical solution of the first embodiment of the present invention is as follows: a domain generalization target detection method based on a large model guidance, the method comprising: acquiring a preset training image set and an image to be predicted; wherein, the training image set includes multiple training images, the image content types of different training images are the same, and the image content type of the image to be predicted is the same as the image content type of the training image set; performing foreground mask guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; performing style fusion on the foreground-enhanced image set and the image to be predicted to obtain a style-fused feature map; performing hypergraph domain invariant feature encoding on the style-fused feature map to obtain a feature-encoded feature map; and decoding and detecting the feature-encoded feature map to obtain the target detection result of the image to be predicted.

[0005] Preferably, the step of performing foreground mask guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set includes: obtaining a first image feature of each training image in the training image set, and obtaining a cue feature of each training image in the training image set; decoding the first image feature and the cue feature of each training image to obtain a foreground mask of each training image; and performing a dot product between the foreground mask of each training image and the training image to obtain a foreground-enhanced feature map corresponding to each training image.

[0006] Preferably, the step of style-fusing the foreground-enhanced image set with the image to be predicted to obtain a style-fused feature map includes: extracting first statistical information from the foreground-enhanced image set, the first statistical information including a first mean, a first variance, and a first skewness of pixels; extracting second statistical information from the image to be predicted, the second statistical information including a second mean, a second variance, and a second skewness of pixels; calculating the Vasastan distance between the image to be predicted and the foreground-enhanced image set based on the first and second statistical information; averaging all Vasastan distances to obtain a Vasastan distance mean; converting the Vasastan distance mean into a weight value according to a preset Vasastan distance and weight conversion rule; using the weight value to perform a weighted calculation on the first statistical information to obtain target statistical information; the target statistical information including a target mean, a target variance, and a target skewness; fusing the target statistical information and the second statistical information of the image to be predicted with the image to be predicted to obtain the style-fused feature map.

[0007] Preferably, the feature map after style fusion is obtained using the following formula:

[0008] in, This is the feature map after style fusion. The image to be predicted, This is the second mean. This is the second variance. This is the second skewness. The target mean, Let Variance be the target variance. The target skewness is denoted as .

[0009] Preferably, the step of performing hypergraph domain invariant feature encoding on the style-fused feature map to obtain a feature-encoded feature map includes: performing three downsampling operations on the style-fused feature map to obtain a first downsampled feature map, a second downsampled feature map, and a third downsampled feature map with different resolutions; the third downsampled feature map has the lowest resolution, and the first downsampled feature map has the highest resolution; performing hypergraph feature enhancement and vertex feature extraction on the third downsampled feature map to obtain the vertex features of the third downsampled feature map; generating a hyperedge of the third downsampled feature map; collecting the vertex features through the hyperedge, and linearly projecting the hyperedge after collecting the vertices to obtain hyperedge features; fusing the hyperedge features, the first downsampled feature map, and the second downsampled feature map to obtain the feature-encoded feature map.

[0010] Preferably, the step of performing hypergraph feature enhancement and vertex feature extraction on the third downsampled feature map to obtain the vertex features of the third downsampled feature map includes: performing position embedding on the third downsampled feature map to obtain a position-embedded feature map; extracting second image features from the position-embedded feature map using a multi-head self-attention mechanism; flattening the second image features to obtain flattened features; and performing vertex feature extraction on the flattened features to obtain the vertex features of the third downsampled feature map.

[0011] Preferably, generating the superedge of the third downsampled feature map includes: performing max pooling on the feature map after position embedding to obtain a first output result, and performing average pooling on the feature map after position embedding to obtain a second output result; concatenating and mapping the first output result and the second output result to obtain a first dynamic offset; adding the first dynamic offset to the third downsampled feature map to obtain a first branch output result; performing projection on the flattened feature to obtain a second branch output result; performing max pooling on the flattened feature to obtain a third output result, and performing average pooling on the flattened feature to obtain a fourth output result; concatenating and mapping the third output result and the fourth output result to obtain a second dynamic offset; adding the second dynamic offset to the third downsampled feature map to obtain a third branch output result; multiplying the first branch output result and the second branch output result to obtain a first product result; multiplying the third branch output result and the second branch output result to obtain a second product result; and fusing the first product result and the second product result to obtain the superedge of the third downsampled feature map.

[0012] The specific technical solution of the second embodiment of the present invention is as follows: a domain generalization target detection system based on a large model guidance, the system comprising: an image acquisition module, a foreground mask guidance module, a feature style mapping module, a hypergraph domain invariant feature encoding module, and a decoding and detection module; the image acquisition module is used to acquire a preset training image set and an image to be predicted; wherein, the training image set includes multiple training images, different training images have the same image content type, and the image content type of the image to be predicted is the same as the image content type of the training image set; the foreground mask guidance module is used to perform foreground mask guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; the feature style mapping module is used to perform style fusion on the foreground-enhanced image set and the image to be predicted to obtain a style-fused feature map; the hypergraph domain invariant feature encoding module is used to perform hypergraph domain invariant feature encoding on the style-fused feature map to obtain a feature-encoded feature map; the decoding and detection module is used to decode and detect the feature-encoded feature map to obtain the target detection result of the image to be predicted.

[0013] The specific technical solution of the third embodiment of the present invention is as follows: a domain generalization target detection device based on a large model guide, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method as described in any one of the first embodiments of this application.

[0014] The specific technical solution of the third embodiment of the present invention is as follows: a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the processor performs the steps of the method as described in any one of the first embodiments of this application.

[0015] Implementing the embodiments of the present invention will have the following beneficial effects: This invention acquires a training image set with identical content and combines it with foreground masking to guide the generation of a foreground-enhanced image set. This not only enriches the diversity of data enhancement from the source by leveraging multiple training data, but also precisely focuses on the target foreground for feature enhancement through foreground masking. While improving the richness of data distribution, it effectively preserves the target's localization features and avoids compromising target localization accuracy. Feature adaptation is achieved through style fusion between the foreground-enhanced image set and the image to be predicted. Then, hypergraph domain-invariant feature encoding is used to obtain a feature map with encoded features. This encoded feature map, while retaining key feature information, efficiently extracts domain-invariant features with strong generalization capabilities, avoiding feature information loss and improving feature representation ability, thereby obtaining accurate target detection results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the steps of a first embodiment of a domain generalization target detection method guided by a large model. Figure 2 This is a flowchart illustrating the steps of a second embodiment of a domain generalization target detection method guided by a large model. Figure 3 This is a schematic diagram of the structure of a domain generalization target detection system guided by a large model. Among them, 201 is the image acquisition module; 202 is the foreground mask guidance module; 203 is the feature style mapping module; 204 is the supergraph domain invariant feature encoding module; and 205 is the decoding and detection module. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to such processes, methods, products, or apparatus.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] The unique feature of SAR images lies in their active microwave imaging mechanism, which is unaffected by weather and lighting conditions such as clouds, fog, rain, snow, and day / night cycles. They can penetrate some vegetation and ground cover, continuously acquiring Earth observation data in complex environments. SAR images contain radar scattering characteristics of targets, enabling accurate detection of naval targets based on differences in radar scattering between different objects. This strong environmental adaptability and target penetration capability makes SAR images perform exceptionally well in target detection in adverse weather or lighting conditions. However, due to the scarcity of SAR datasets and the significant domain differences between different SAR datasets, a single-source domain generalization method is needed.

[0022] However, the generalization ability of existing single-source domain generalized ship target detection models for visible light images is severely limited when there are significant domain differences between training and testing data. Existing methods face three core technical challenges in practical applications. First, the large differences in the relative relationships between ship targets and backgrounds between the SAR domain and the visible light domain, as well as between different SAR domain datasets, lead to cross-domain knowledge transfer bias. Second, the large texture differences between visible light and SAR datasets, as well as between different SAR datasets, result in insufficient generalization performance of traditional methods in cross-modal generalization tasks. Finally, extracting domain-invariant features of ship targets in the visible light domain and different SAR domain datasets is difficult, leading to insufficient robustness of features extracted by traditional methods. Therefore, researching robust single-source domain generalized target detection in complex SAR scenes has high practical significance and research value. Single-source domain generalized target detection models for visible light images cannot achieve satisfactory performance on SAR target domain datasets. Therefore, target detection methods for the SAR target domain have been rapidly developed. Developing an efficient and high-precision single-source domain generalized SAR target detection method is of great significance, but it remains a huge challenge for scenarios that simultaneously include visible light and SAR.

[0023] To resolve the above issues, please refer to [link / reference]. Figure 1 This is a flowchart illustrating the steps of a domain generalization target detection method based on a large model guided by the first embodiment of this application. By combining SAM2 foreground mask guidance, feature style mapping layer, and hypergraph domain invariant feature encoder, it ultimately achieves cross-domain single-source domain generalization ship target detection, breaking through the application bottleneck of existing technologies, thereby obtaining accurate target detection results. The method includes: Step 101: Obtain a preset training image set and an image to be predicted; wherein, the training image set includes multiple training images, and the image content of different training images is the same, and the image content of the image to be predicted is the same as the image content of the training image set; Step 102: Perform foreground masking on the training image set to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; Step 103: Perform style fusion between the foreground-enhanced image set and the image to be predicted to obtain a style-fused feature map; Step 104: Perform hypergraph domain invariant feature encoding on the style-fused feature map to obtain the feature-encoded feature map; Step 105: Decode and detect the feature map after feature encoding to obtain the target detection result of the image to be predicted.

[0024] Specifically, a pre-defined training image set and a prediction image are acquired. For example, in a maritime traffic scenario, the training image set contains n images of ships from different angles and under different lighting conditions, all depicting ships. The prediction image is also a ship image, and its content is consistent with the training image set. Foreground masking is applied to the training image set to generate a foreground mask for each training image. The training images are then enhanced based on the foreground mask, highlighting the ship and suppressing background information, thus obtaining a foreground enhancement feature map for each training image. These feature maps constitute the foreground-enhanced image set. The foreground-enhanced image set and the prediction image are then style-fused, extracting the style features of the foreground-enhanced image set and the content features of the prediction image. These are then fused to generate a style-fused feature map, allowing the prediction image to retain its ship content while possessing the style characteristics of the training image set. The style-fused feature map is subjected to hypergraph-invariant feature encoding to obtain a feature-encoded feature map. This feature map is then decoded and detected, restoring the image features to a visual representation. Object detection algorithms, such as the YOLO series, are then used to detect ship targets in the image to be predicted, outputting the target detection results, including the ship's position and category. Alternatively, the image to be predicted can be multiplied by a foreground mask set to 1 to obtain the original feature map, which is then used as the image to be predicted to perform the above steps.

[0025] By acquiring a training image set with identical content and combining it with foreground masking to guide the generation of a foreground-enhanced image set, the diversity of data augmentation is enriched from the root by leveraging multiple training data. Furthermore, the foreground masking precisely focuses on the target foreground for feature enhancement, improving the richness of data distribution while effectively preserving the target's localization features and avoiding any compromise to target localization accuracy. Feature adaptation is achieved through style fusion between the foreground-enhanced image set and the image to be predicted. Then, hypergraph domain-invariant feature encoding is used to obtain a feature map with encoded features. This encoded feature map, while retaining key feature information, efficiently extracts domain-invariant features with strong generalization capabilities, avoiding feature information loss and improving feature representation capabilities, thereby obtaining accurate target detection results.

[0026] In a specific embodiment, the step of performing foreground mask guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set includes: obtaining a first image feature of each training image in the training image set, and obtaining a cue feature of each training image in the training image set; decoding the first image feature and the cue feature of each training image to obtain a foreground mask of each training image; and performing a dot product between the foreground mask of each training image and the training image to obtain a foreground-enhanced feature map corresponding to each training image.

[0027] Specifically, the training image and its bounding box are input into the SAM2 foreground mask-guided image encoder and cue encoder, respectively, and the encoder outputs image features and cue features, respectively. The image features output by the image encoder and the cue features output by the cue encoder are both input into the mask decoder for processing to generate the foreground mask for the training set. The generated foreground mask for the training set is then multiplied by the input training image to obtain the foreground enhancement feature map.

[0028] In a specific embodiment, the step of style-fusing the foreground-enhanced image set with the image to be predicted to obtain a style-fused feature map includes: extracting first statistical information from the foreground-enhanced image set, the first statistical information including a first mean, a first variance, and a first skewness of pixels; extracting second statistical information from the image to be predicted, the second statistical information including a second mean, a second variance, and a second skewness of pixels; calculating the Vasastan distance between the image to be predicted and the foreground-enhanced image set based on the first and second statistical information; averaging all Vasastan distances to obtain a Vasastan distance mean; converting the Vasastan distance mean into a weight value according to a preset Vasastan distance and weight conversion rule; using the weight value to perform a weighted calculation on the first statistical information to obtain target statistical information; the target statistical information including a target mean, a target variance, and a target skewness; and fusing the target statistical information and the second statistical information of the image to be predicted with the image to be predicted to obtain the style-fused feature map.

[0029] Specifically, after acquiring the foreground-enhanced image set, pixel-level statistics are performed on each image in the set. For each image, the first mean of all pixels is calculated, i.e., the average value of all pixel values; the first variance is calculated, measuring the dispersion of pixel values ​​around the mean; and the first skewness is calculated, describing the asymmetry of the pixel value distribution. This yields the first statistical information of the foreground-enhanced image set, including the first mean, first variance, and first skewness for each image. The same pixel-level statistical operation is performed on the ordinary landscape photograph image to be predicted, calculating the second mean, second variance, and second skewness of all pixels to obtain the second statistical information of the image to be predicted. For each image in the foreground-enhanced image set, the Vasasstein distance between them is calculated based on their first statistical information and the second statistical information of the image to be predicted. The Vasasstein distance measures the difference between two probability distributions; here, it is used to measure the difference between the statistical features of images of different styles. The average of all calculated Vasasstein distances is obtained, which reflects the average difference in statistical features between the image to be predicted and the entire foreground-enhanced image set. According to the preset Vasastan distance and weight transformation rules, the Vasastan distance mean is converted into a weight value. The transformed weight value is used to perform a weighted calculation on the first statistical information of the foreground enhancement image set. The target statistical information and the second statistical information of the image to be predicted are fused with the image to be predicted. By adjusting the pixel values ​​of the image, its statistical features gradually approach the target statistical information, while retaining some original features of the image to be predicted. Finally, a style-fused feature map is obtained, which has both impressionistic style characteristics and retains some content information of the original image to be predicted.

[0030] Specifically, the Vasastan distance is obtained using the following formula: ,in, The distance to Vasastan is given. This is the second mean. This is the second variance. This is the second skewness. The first mean, The first variance, This is the first skewness.

[0031] Specifically, the target mean, target variance, and target skewness are obtained using the following formulas:

[0032] in, The target mean, Let Variance be the target variance. The target skewness, The mean of the Vassastan distance, For the weight, The first mean, The first variance, This is the first skewness.

[0033] In a specific embodiment, the feature map after style fusion is obtained using the following formula:

[0034] in, This is the feature map after style fusion. The image to be predicted, This is the second mean. This is the second variance. This is the second skewness. The target mean, Let Variance be the target variance. The target skewness is denoted as .

[0035] In a specific embodiment, the step of performing hypergraph domain invariant feature encoding on the style-fused feature map to obtain a feature-encoded feature map includes: performing downsampling on the style-fused feature map three times to obtain a first downsampled feature map, a second downsampled feature map, and a third downsampled feature map with different resolutions; the third downsampled feature map has the lowest resolution, and the first downsampled feature map has the highest resolution; performing hypergraph feature enhancement and vertex feature extraction on the third downsampled feature map to obtain the vertex features of the third downsampled feature map; generating a hyperedge of the third downsampled feature map; collecting the vertex features through the hyperedge, and linearly projecting the hyperedge after collecting the vertices to obtain hyperedge features; fusing the hyperedge features, the first downsampled feature map, and the second downsampled feature map to obtain the feature-encoded feature map.

[0036] Specifically, the feature map after style fusion is downsampled three times to obtain a first downsampled feature map S3, a second downsampled feature map S4, and a third downsampled feature map S5. Hypergraph feature enhancement and vertex feature extraction are performed on the third downsampled feature map S5 to obtain the vertex features of the third downsampled feature map S5. Then, the hyperedge of the third downsampled feature map S5 is generated through three parallel branches. The features of all vertices are collected through the hyperedge, and the hyperedge features are obtained through linear projection. The hyperedge features, the first downsampled feature map S3, and the second downsampled feature map S4 are fused to obtain the feature map after feature encoding. Specifically, the hyperedge features are propagated back to the corresponding vertices to update the feature representation of the vertices.

[0037] In a specific embodiment, the step of performing hypergraph feature enhancement and vertex feature extraction on the third downsampled feature map to obtain the vertex features of the third downsampled feature map includes: performing position embedding on the third downsampled feature map to obtain a position-embedded feature map; extracting a second image feature from the position-embedded feature map using a multi-head self-attention mechanism; flattening the second image feature to obtain flattened features; and performing vertex feature extraction on the flattened features to obtain the vertex features of the third downsampled feature map.

[0038] Specifically, hypergraph feature enhancement is performed on the S5 feature map. First, 2D position embedding is added, then features are extracted through a multi-head self-attention mechanism, and then the features are flattened and vertex features are extracted to obtain the features of all vertices in the feature map.

[0039] In a specific embodiment, generating the hyperedge of the third downsampled feature map includes: performing max pooling on the feature map after position embedding to obtain a first output result, and performing average pooling on the feature map after position embedding to obtain a second output result; concatenating and mapping the first output result and the second output result to obtain a first dynamic offset; adding the first dynamic offset to the third downsampled feature map to obtain a first branch output result; performing projection on the flattened feature to obtain a second branch output result; performing max pooling on the flattened feature to obtain a third output result, and performing average pooling on the flattened feature to obtain a fourth output result; concatenating and mapping the third output result and the fourth output result to obtain a second dynamic offset; adding the second dynamic offset to the third downsampled feature map to obtain a third branch output result; multiplying the first branch output result and the second branch output result to obtain a first product result; multiplying the third branch output result and the second branch output result to obtain a second product result; and fusing the first product result and the second product result to obtain the hyperedge of the third downsampled feature map.

[0040] Specifically, the hyperedge is generated through three parallel branches: The first branch performs position embedding, followed by max pooling and average pooling, then concatenates the outputs of these two pooling operations to map a dynamic offset, which is then added to the global prototype to obtain the output; the second branch performs projection operations on the flattened features; the third branch directly performs max pooling and average pooling on the flattened features, concatenates the outputs of these two pooling operations to map a dynamic offset, and then adds it to the global prototype to obtain the output. The outputs of the first and third branches are multiplied by the output of the second branch, and then fused with learnable features to finally generate the hyperedge.

[0041] In a specific embodiment, please refer to Figure 2 The complete embodiment of this application is as follows: (1) SAM2 foreground mask guidance (5 groups in total) Training phase: 1.1) Input the images and bounding boxes of the training set into the SAM2 foreground mask-guided image encoder and cue encoder respectively, and output the image features and cue features respectively.

[0042] 1.2) The image features output by the image encoder and the cue features output by the cue encoder are all input into the mask decoder for processing to generate the foreground mask of the training set.

[0043] 1.3) The foreground mask of the generated training set is multiplied by the image of the input training set to obtain the feature map for foreground enhancement. Reasoning stage: 1.1) During the testing phase, disable the image encoder, prompt encoder, and mask decoder, and set all foreground masks to 1.

[0044] 1.2) Perform a dot product between the foreground mask (set to 1) and the input test set image to output the original feature map.

[0045] (2) Feature style mapping layer (5 groups in total) Training phase: 2.1) Input the feature map for foreground enhancement and extract third-order statistical features by channel dimension, including the mean, variance, and skewness of pixels.

[0046] 2.2) Establish a "learnable style prototype library" containing multiple "style prototypes," each corresponding to a set of mean, variance, and skewness to represent different style types. During training, these prototypes are continuously optimized through momentum updates using statistical information from new samples, allowing them to gradually accumulate the style characteristics of the dataset. When processing new samples, the differences (distances) between the current sample's statistical information and each prototype in the library are calculated, and these differences are then converted into weights to measure the influence of each prototype on the current sample.

[0047] 2.3) Based on the weights obtained in the previous step, the statistical information (mean, variance, skewness) of all prototypes in the prototype library is weighted and mixed to obtain the "target style statistics" that specifically match the current sample.

[0048] 2.4) Adaptive style fusion and feature renormalization are achieved through weighted aggregation.

[0049] Reasoning stage: 2.1) Input the original feature map and extract third-order statistical features by channel dimension, including mean, variance and skewness.

[0050] 2.2) Fix the “target style statistics” and achieve adaptive style fusion and feature renormalization through weighted aggregation.

[0051] (3) Hypergraph Invariant Feature Encoder 3.1) Receive the downsampled feature maps (S3, S4, S5) of three resolutions output by the last three feature style mapping layers.

[0052] 3.2) Perform hypergraph feature enhancement on the S5 feature map. First, add 2D position embedding, then extract features through multi-head self-attention mechanism, and then flatten the features to obtain the features of all vertices in the feature map.

[0053] 3.3) Then, the hyperedge is generated through three parallel branches: The first branch first performs position embedding, then performs max pooling and average pooling respectively, then concatenates the outputs of these two pooling operations to map to a dynamic offset, and finally adds it to the global prototype to obtain the output; The second branch performs a projection operation on the flattened features; The third branch directly performs max pooling and average pooling on the flattened features, then concatenates the outputs of these two pooling operations to map them into a dynamic offset, and finally adds it to the global prototype to obtain the output.

[0054] The outputs of the first and third branches are multiplied by the output of the second branch, and then fused with learnable features to finally generate the hyperedge.

[0055] 3.4) Collect the features of all vertices through the hyperedge and obtain the hyperedge features through linear projection.

[0056] 3.5) Propagate the hyperedge features back to the corresponding vertices and update the feature representations of the vertices.

[0057] (4) Decoder and detection head The decoder uses the feature map output by the encoder to perform cross-attention interaction and output query features. The detection head then directly predicts the target category and bounding box based on the decoder's output, completing the ship target detection.

[0058] By introducing the SAM2 foreground mask, the cross-domain knowledge transfer bias caused by the large difference in the relative relationship between the target and the background is effectively mitigated. The design of the feature style mapping layer addresses the issue of insufficient generalization performance of traditional methods on cross-modal generalization tasks due to large texture differences between visible light and SAR datasets, as well as between different SAR datasets. Furthermore, this embodiment utilizes a hypergraph-invariant feature enhancement encoder to effectively extract and enhance domain-invariant features. Compared to other methods, the proposed method exhibits better generalization performance.

[0059] This embodiment, through an innovative combination of SAM2 foreground mask guidance, feature style mapping layer, and supergraph domain invariant feature encoder, breaks through the core bottlenecks of traditional SAR ship detection technology in cross-domain generalization, data dependence, and detection accuracy. Combined with the inherent advantages of SAR imaging such as "all-weather, all-time", it can be widely applied to core areas such as maritime defense security, maritime emergency search and rescue, maritime traffic and port management, fishery resources and marine ecological supervision, and the protection of maritime rights, providing high-precision and efficient ship detection solutions for various scenarios. It can also be extended to other SAR target detection scenarios, driving the development of the entire industry chain and helping my country's SAR industry upgrade from "equipment localization" to "algorithm autonomy and intelligence", possessing extremely high market potential and social value.

[0060] In this embodiment, the SAM2 foreground mask guidance can be replaced by the corresponding structure of other zero-shot semantic segmentation methods, the feature style mapping layer can be replaced by the corresponding part of other feature style transfer methods, and the hypergraph domain invariant feature enhancement encoder can be replaced by combining other domain invariant feature extraction methods with the encoder.

[0061] In a specific embodiment, please refer to Figure 3 This is a schematic diagram of a domain generalization target detection system based on a large model guided by the second embodiment of this application. The system includes: an image acquisition module 201, a foreground mask guidance module 202, a feature style mapping module 203, a hypergraph domain invariant feature encoding module 204, and a decoding and detection module 205. The image acquisition module 201 is used to acquire a preset training image set and an image to be predicted. The training image set includes multiple training images, and the image content types of the different training images are the same. The image content type of the image to be predicted is the same as that of the training image set. The foreground mask guidance module 202 is used to perform image processing on the training image set. The image set is guided by foreground masking to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; the feature style mapping module 203 is used to perform style fusion between the foreground-enhanced image set and the image to be predicted to obtain a style-fused feature map; the hypergraph invariant feature encoding module 204 is used to perform hypergraph invariant feature encoding on the style-fused feature map to obtain a feature-encoded feature map; the decoding and detection module 205 is used to decode and detect the feature-encoded feature map to obtain the target detection result of the image to be predicted.

[0062] In a specific embodiment, the third embodiment of this application provides a domain generalization target detection device based on a large model, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method as described in any one of the first embodiments of this application.

[0063] In a specific embodiment, the fourth embodiment of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method as described in any one of the first embodiments of this application.

[0064] The above embodiments merely illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A domain generalization target detection method based on a large model, characterized in that, The method includes: Obtain a preset training image set and an image to be predicted; wherein, the training image set includes multiple training images, the different training images have the same image content type, and the image content type of the image to be predicted is the same as the image content type of the training image set; Foreground masking is applied to the training image set to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; The foreground-enhanced image set is style-fused with the image to be predicted to obtain a style-fused feature map; The feature map after style fusion is subjected to hypergraph domain invariant feature encoding to obtain the feature map after feature encoding; The feature map after feature encoding is decoded and detected to obtain the target detection result of the image to be predicted.

2. The domain generalization target detection method based on large model guidance as described in claim 1, characterized in that, The step of performing foreground masking guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set includes: Obtain the first image feature of each training image in the training image set, and obtain the cue feature of each training image in the training image set; The foreground mask of each training image is obtained by decoding the first image feature and the cue feature of each training image. The foreground mask of each training image is multiplied by the training image to obtain the foreground enhancement feature map corresponding to each training image.

3. The domain generalization target detection method based on large model guidance as described in claim 1, characterized in that, The step of style-fusing the foreground-enhanced image set with the image to be predicted to obtain a style-fused feature map includes: Extract first statistical information from the foreground-enhanced image set, the first statistical information including the first mean, first variance, and first skewness of the pixels; Extract the second statistical information of the image to be predicted, the second statistical information including the second mean, second variance and second skewness of the pixels; The Vasasstein distance between the image to be predicted and the foreground-enhanced image set is calculated based on the first statistical information and the second statistical information. The average of all Vasastan distances is calculated to obtain the Vasastan distance mean. According to the preset Vasastan distance and weight conversion rules, the mean Vasastan distance is converted into a weight value; The first statistical information is weighted using the weight values ​​to obtain target statistical information; the target statistical information includes target mean, target variance, and target skewness. The target statistical information and the second statistical information of the image to be predicted are fused with the image to be predicted to obtain the feature map after style fusion.

4. The domain generalization target detection method based on large model guidance as described in claim 3, characterized in that, The feature map after style fusion is obtained using the following formula: in, This is the feature map after style fusion. The image to be predicted, This is the second mean. This is the second variance. This is the second skewness. The target mean, Let Variance be the target variance. The target skewness is denoted as .

5. The domain generalization target detection method based on a large model as described in claim 1, characterized in that, The step of performing hypergraph-invariant feature encoding on the style-fused feature map to obtain the feature-encoded feature map includes: The style-fused feature map is downsampled three times to obtain a first downsampled feature map, a second downsampled feature map, and a third downsampled feature map with different resolutions; the third downsampled feature map has the lowest resolution, and the first downsampled feature map has the highest resolution. Hypergraph feature enhancement and vertex feature extraction are performed on the third downsampled feature map to obtain the vertex features of the third downsampled feature map; Generate the superedge of the third downsampled feature map; The features of the vertices are collected through the hyperedge, and the hyperedge after collecting the vertices is linearly projected to obtain the hyperedge features; The super-edge feature, the first downsampled feature map, and the second downsampled feature map are fused to obtain the feature map after feature encoding.

6. The domain generalization target detection method based on large model guidance as described in claim 5, characterized in that, The step of performing hypergraph feature enhancement and vertex feature extraction on the third downsampled feature map to obtain the vertex features of the third downsampled feature map includes: The third downsampled feature map is embedded with a location to obtain a feature map with the location embedded. The second image features are extracted from the feature map after the location embedding using a multi-head self-attention mechanism; The second image feature is flattened to obtain the flattened feature; Vertex feature extraction is performed on the flattened features to obtain the vertex features of the third downsampled feature map.

7. The domain generalization target detection method based on large model guidance as described in claim 5, characterized in that, The generation of the hyperedge of the third downsampled feature map includes: Max pooling is performed on the feature map after the location embedding to obtain a first output result, and average pooling is performed on the feature map after the location embedding to obtain a second output result. The first output result and the second output result are concatenated and mapped to obtain a first dynamic offset. The first dynamic offset is added to the third downsampled feature map to obtain a first branch output result. The second branch output result is obtained by projecting the flattened features. Max pooling is performed on the flattened features to obtain a third output result, and average pooling is performed on the flattened features to obtain a fourth output result. The third output result and the fourth output result are concatenated and mapped to obtain a second dynamic offset. The second dynamic offset is added to the third downsampled feature map to obtain a third branch output result. Multiply the output of the first branch by the output of the second branch to obtain the first product result, and multiply the output of the third branch by the output of the second branch to obtain the second product result; The first product result is fused with the second product result to obtain the superedge of the third downsampled feature map.

8. A domain generalization target detection system based on a large model, characterized in that, The system includes: an image acquisition module, a foreground mask guidance module, a feature style mapping module, a supergraph domain invariant feature encoding module, and a decoding and detection module; The image acquisition module is used to acquire a preset training image set and an image to be predicted; wherein, the training image set includes multiple training images, and the image content of different training images is the same type, and the image content of the image to be predicted is the same type as the image content of the training image set; The foreground masking guidance module is used to perform foreground masking guidance on the training image set to obtain a foreground-enhanced image set corresponding to the training image set; wherein, the foreground-enhanced image set includes a foreground-enhanced feature map corresponding to each training image; The feature style mapping module is used to perform style fusion between the foreground-enhanced image set and the image to be predicted to obtain a style-fused feature map. The hypergraph domain invariant feature encoding module is used to perform hypergraph domain invariant feature encoding on the style fusion feature map to obtain the feature map after feature encoding. The decoding and detection module is used to decode and detect the feature map after feature encoding to obtain the target detection result of the image to be predicted.

9. A domain generalization target detection device based on a large model, comprising a memory and a processor, characterized in that, The memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 7.

Images