An infrared dim small target ROI segmentation method, device and equipment based on a priori gating adapter and a storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for infrared weak target ROI segmentation based on a priori gating adapter, belonging to the field of computer vision technology. The method includes: constructing an image encoder by embedding a priori gating adapter under the condition of freezing the backbone parameters of the basic visual model; introducing priors such as local contrast of the weak target, background texture complexity, and edge gradient into feature extraction to achieve lightweight adaptation; generating a coarse segmentation result of the whole image based on multi-scale features; correcting it using a boundary enhancement residual refinement module to obtain a boundary enhancement segmentation result; generating a saliency probability map; determining the ROI region through thresholding and pooling; cropping the original image, probability map, and edge map according to the determined ROI region to form a patch, inputting it into a fine segmentation network to obtain patch prediction results; finally, pasting the patch prediction results back into the original image coordinate system; and weighted fusing of overlapping areas to obtain the final segmentation result, thereby improving the segmentation accuracy of infrared weak targets.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and storage medium for infrared weak target ROI segmentation based on a priori gating adapter. Background Technology

[0002] Currently, infrared detection and identification of small targets has significant application value in low-altitude security and disaster relief missions, and is one of the key technologies for achieving all-weather, long-range target perception. Compared to visible light imaging, infrared imaging does not rely on natural light, has the advantages of being applicable day and night, and can penetrate smoke and fog to a certain extent, thus having stronger engineering practicality in complex environments. However, the scale of infrared small targets often occupies only a few pixels, with severely insufficient structural information and texture details, sometimes even appearing only as near-point bright spots; at the same time, targets are often in dynamic and complex backgrounds, such as ground thermal radiation fluctuations, cloud / sea surface disturbances, etc., which introduce significant interference, causing the target and background distribution to highly overlap, thus posing a significant challenge to fine segmentation.

[0003] Existing infrared weak target segmentation methods can be broadly categorized into two types: traditional model-driven methods and deep learning methods. Traditional methods typically revolve around "background suppression—target enhancement—threshold decision," with representative methods including multi-scale filtering, local contrast enhancement, morphological operators, and CFAR-like detection. Additionally, there are low-rank-sparse decomposition methods that model the background as a low-rank structure and the target as a sparse anomaly, as well as trajectory / cumulative detection methods that leverage multi-frame spatiotemporal consistency to improve detectability. These methods offer advantages such as strong interpretability and low dependence on annotations, but they generally suffer from parameter sensitivity, insufficient robustness to complex backgrounds and noise conditions, and limited cross-scene generalization capabilities. They are particularly prone to false alarms in scenarios with prominent false targets such as strong texture edges, sea-line terrain, clouds, and thermal noise, and struggle to simultaneously achieve both detection and high-quality boundary segmentation.

[0004] Deep learning methods have been widely applied in infrared weak target tasks in recent years. Common approaches include detection and segmentation networks based on multi-scale feature fusion such as U-Net / FPN, feature enhancement methods incorporating attention mechanisms, and representation learning frameworks that utilize Transformer for global dependency modeling. Compared to traditional methods, deep learning can learn more discriminative feature representations through data-driven learning and mitigate interference from complex backgrounds to some extent. However, the inherent characteristics of infrared weak targets, such as insufficient information, extreme imbalance between positive and negative samples, and significant distribution drift across sensors / scenes, make network models prone to overfitting and undergeneralization of specific data distributions. Furthermore, small targets are easily submerged during downsampling, tokenization, and multi-layer feature aggregation, leading to decreased recall or segmentation quality issues such as bloated or fragmented boundaries.

[0005] With the development of self-supervised pre-training and visual foundational models, feature representations based on large-scale pre-training have demonstrated strong transfer capabilities in downstream tasks. Addressing the scarcity of infrared weak target annotations and the variability of scenes, efficient parameter fine-tuning (PEFT) has become an important direction. Adapters, by inserting lightweight trainable modules while freezing the backbone network, achieve task adaptation and rapid deployment with fewer parameters. However, general-purpose adapters often perform feature updates without prior constraints by "learning residuals," often lacking the explicit prior integration and controllable suppression mechanisms required for infrared weak targets. This makes it difficult to stably distinguish between real weak targets and bright noise / local textured false targets, and also makes it difficult to effectively reduce false alarms and improve boundary accuracy while ensuring recall.

[0006] As can be seen from the above, improving the segmentation accuracy of infrared weak targets in complex backgrounds during the infrared weak target ROI segmentation process based on prior gating adapters is an urgent problem to be solved. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for infrared weak target ROI segmentation based on a priori gating adapters. This method improves the segmentation accuracy of infrared weak targets in complex backgrounds and enhances the target boundary representation effect during the infrared weak target ROI segmentation process based on a priori gating adapters, thereby improving the recognition capability of low signal-to-noise ratio, small-scale targets. The specific solution is as follows: Firstly, this application provides a method for infrared weak target ROI segmentation based on a priori gating adapter, including: Infrared images are input into a pre-trained visual base model encoder embedded with a priori gating adapter. Feature extraction is performed under the condition that the backbone parameters of the pre-trained visual base model are frozen to obtain the corresponding multi-scale features. The multi-scale features are input into the global segmentation head for feature fusion and segmentation prediction, and the segmentation results are upsampled to the original image resolution to obtain the full-image coarse segmentation prediction results. A grayscale intensity map is generated based on the infrared image, and a first edge map is calculated based on the grayscale intensity map using an edge detection operator. The coarse segmentation prediction result of the whole image, the grayscale intensity map, and the first edge map are stitched together in the channel dimension, and the resulting stitched feature map is input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result. A saliency probability map is generated based on the boundary enhancement segmentation result, and threshold filtering and pooling operations are performed on the saliency probability map to determine the candidate Region of Interest (ROI). Based on the ROI, the infrared image, the saliency probability map, and the corresponding edge map are cropped to obtain image patches, guide probability patches, and edge patches. The image patch, the guiding probability patch, and the edge patch are concatenated along the channel dimension, and the concatenation result is upsampled and then input into the patch-level fine segmentation network to obtain the corresponding segmentation prediction patch. Based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in each mapping result are weighted and fused to obtain the infrared weak target segmentation result.

[0008] Optionally, the prior gating adapter is embedded in the pre-trained visual base model encoder to generate gating coefficients based on the prior information of contrast, edge intensity, and background complexity of the infrared image, and to perform lightweight fine-tuning of the features in the encoder based on the gating coefficients.

[0009] Optionally, the step of concatenating the full-image coarse segmentation prediction result, the grayscale intensity map, and the first edge map along the channel dimension, and inputting the resulting concatenated feature map into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result, includes: The coarse segmentation prediction result of the whole image, the grayscale intensity map and the first edge map are concatenated in the channel dimension to obtain the concatenated feature map. In the boundary enhancement residual refinement module, the stitched feature map is input into a residual network consisting of multiple convolutions, normalization, and nonlinear activations, so as to use the residual network to predict the residual terms corresponding to the stitched feature map. The coarse segmentation prediction result of the whole image is corrected based on the residual term to obtain the boundary enhancement segmentation result.

[0010] Optionally, the step of generating a saliency probability map based on the boundary enhancement segmentation result, and performing threshold filtering and pooling operations on the saliency probability map to determine candidate Regions of Interest (ROIs); and cropping the infrared image, the saliency probability map, and the corresponding edge map based on the ROIs to obtain image patches, guiding probability patches, and edge patches, including: The boundary enhancement segmentation results are mapped using a sigmoid function to obtain a saliency probability map; The saliency probability map is pooled according to a preset step size to obtain a probability response map; The probability response map is subjected to threshold filtering, and the pixel positions with response values ​​greater than a preset threshold are determined as candidate ROI centers; When the number of candidate ROI centers is greater than the preset number, the candidate ROI centers are sorted from high to low according to the response value, so as to select a preset number of ROI centers from the sorting results. Based on the centers of each ROI, a fixed-size ROI region is generated, and based on the ROI region, the infrared image, the saliency probability map, and the corresponding edge map are cropped by image patching and probability patching to obtain image patch, guide probability patch, and edge patch.

[0011] Optionally, the process of obtaining the edge patch includes: The probability map is obtained by performing an S-shaped function transformation on the guiding probability patch. The probability map is processed using an edge detection operator to obtain a corresponding second edge map, and an edge patch is generated based on the ROI and the second edge map.

[0012] Optionally, the patch-level fine segmentation network includes a dual-branch coding module, a multi-scale dual-stream fusion module, and a decoding module; The dual-branch coding module includes an independent image coding branch and a guide coding branch. The image coding branch is used to extract image patch features, and the input of the image coding branch is a three-channel image patch formed by copying a single-channel image patch. The guide coding branch is used to extract guide patch features, and the input of the guide coding branch is a three-channel guide patch formed by concatenating a single-channel image patch, a segmentation patch, and an edge patch in the channel dimension. The multi-scale dual-stream fusion module is used to fuse the image patch features and the guide patch features at different scales; The decoding module is used to perform step-by-step upsampling and feature reconstruction on the fused features and output segmentation prediction patches.

[0013] Optionally, based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in each mapping result are weighted and fused to obtain the infrared weak target segmentation result, including: Based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped to the original image coordinate system to obtain several mapping results; When there are overlapping regions in the mapping results, the predicted values ​​at the same pixel position are weighted and fused according to the preset weights to obtain the weighted fusion result. When there are no overlapping regions in the mapping results, the predicted value of the corresponding patch is determined; Infrared weak target segmentation results are generated based on the weighted fusion result and the predicted value.

[0014] Secondly, this application provides an infrared ROI segmentation device for weak targets based on a priori gating adapter, comprising: The multi-scale feature generation module is used to input infrared images into a pre-trained visual base model encoder embedded with a priori gating adapter, and to extract features under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. The full-image coarse segmentation prediction result generation module is used to input the multi-scale features into the global segmentation head for feature fusion and segmentation prediction, and upsample the segmentation result to the original image resolution to obtain the full-image coarse segmentation prediction result. The boundary enhancement segmentation result generation module is used to generate a grayscale intensity map based on the infrared image, and calculate a first edge map based on the grayscale intensity map using an edge detection operator; the full image coarse segmentation prediction result, the grayscale intensity map, and the first edge map are stitched together in the channel dimension, and the resulting stitched feature map is input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result; The ROI determination module is used to generate a saliency probability map based on the boundary enhancement segmentation result, and perform threshold filtering and pooling operations on the saliency probability map to determine the ROI of the candidate region of interest; based on the ROI, the infrared image, the saliency probability map and the corresponding edge map are cropped to obtain image patches, guide probability patches and edge patches. The segmentation prediction patch generation module is used to stitch the image patch, the guiding probability patch and the edge patch in the channel dimension, and input the stitching result into the patch-level fine segmentation network after upsampling to obtain the corresponding segmentation prediction patch. The infrared weak target segmentation result generation module is used to map the segmentation prediction patch back to the original image coordinate system based on the ROI position corresponding to each segmentation prediction patch, and to perform weighted fusion on the overlapping areas in each mapping result to obtain the infrared weak target segmentation result.

[0015] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned infrared weak target ROI segmentation method based on a priori gating adapter.

[0016] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned infrared weak target ROI segmentation method based on a priori gating adapter.

[0017] As can be seen from the above, this application first requires inputting the infrared image into a pre-trained visual base model encoder embedded with a prior gating adapter. Feature extraction is performed under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. The multi-scale features are then input into a global segmentation head for feature fusion and segmentation prediction, and the segmentation result is upsampled to the original image resolution to obtain a coarse full-image segmentation prediction result. A grayscale intensity map is generated based on the infrared image, and a first edge map is calculated based on the grayscale intensity map using an edge detection operator. The coarse full-image segmentation prediction result, the grayscale intensity map, and the first edge map are then concatenated along the channel dimension, and the resulting concatenated feature map is input into a boundary enhancement residual refinement module for residual correction to obtain... Boundary enhancement segmentation results; a saliency probability map is generated based on the boundary enhancement segmentation results, and thresholding and pooling operations are performed on the saliency probability map to determine candidate Regions of Interest (ROIs); based on the ROIs, the infrared image, saliency probability map, and corresponding edge map are cropped to obtain image patches, guiding probability patches, and edge patches; the image patches, guiding probability patches, and edge patches are stitched together along the channel dimension, and the stitched result is upsampled and input into a patch-level fine segmentation network to obtain corresponding segmentation prediction patches; based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in each mapping result are weighted and fused to obtain the infrared weak target segmentation results.

[0018] Therefore, this application first requires inputting the infrared image into a pre-trained visual base model encoder embedded with a priori gating adapter. Feature extraction is performed under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. Secondly, the multi-scale features are input into a global segmentation head for feature fusion and segmentation prediction, and the segmentation result is upsampled to the original image resolution to obtain a coarse full-image segmentation prediction result. A grayscale intensity map is generated based on the infrared image, and a first edge map is calculated based on the grayscale intensity map using an edge detection operator. Then, the coarse full-image segmentation prediction result, the grayscale intensity map, and the first edge map are concatenated along the channel dimension, and the resulting concatenated feature map is input into a boundary enhancement residual refinement module for residual correction to obtain the edge map. The process involves several steps: First, boundary enhancement segmentation results are used. Then, a saliency probability map is generated based on the boundary enhancement segmentation results, and thresholding and pooling operations are performed on the saliency probability map to determine candidate Regions of Interest (ROIs). Based on the ROIs, the infrared image, saliency probability map, and corresponding edge map are cropped to obtain image patches, guiding probability patches, and edge patches. Next, the image patches, guiding probability patches, and edge patches are concatenated along the channel dimension, and the concatenated result is upsampled and input into a patch-level fine-grained segmentation network to obtain corresponding segmentation prediction patches. Finally, based on the ROI positions corresponding to each segmentation prediction patch, the segmentation prediction patches are mapped back to the original image coordinate system, and overlapping areas in the mapping results are weighted and fused to obtain the infrared weak target segmentation result. This approach improves the segmentation accuracy of infrared weak targets in complex backgrounds and enhances the target boundary representation effect during the infrared weak target ROI segmentation process based on a priori gating adapter, thereby improving the recognition capability of low signal-to-noise ratio and small-scale targets. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 This is a flowchart of an infrared ROI segmentation method for small targets based on a priori gating adapter disclosed in this application; Figure 2 This application discloses a specific method for infrared weak target ROI segmentation based on a priori gating adapter. Figure 3 This is a flowchart illustrating the encoder of a specific InfMAE basic model. Figure 4This is a schematic diagram of an infrared ROI segmentation device based on a priori gating adapter disclosed in this application. Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Currently, infrared weak target detection and recognition has significant application value in low-altitude security, disaster relief, and other tasks, and is one of the key technologies for achieving all-weather, long-distance target perception. Compared with visible light imaging, infrared imaging does not rely on natural lighting, has the advantages of being applicable day and night, and can penetrate smoke and fog to a certain extent, thus having stronger engineering practicality in complex environments. However, the scale of infrared weak targets often occupies only a few pixels, with severely insufficient structural information and texture details, sometimes even appearing only as near-point bright spots; at the same time, targets are often in dynamic and complex backgrounds, such as ground thermal radiation fluctuations, cloud / sea surface disturbances, etc., which introduce significant interference, causing the target and background distribution to highly overlap, thus making fine segmentation a significant challenge. To address this, this application provides an infrared weak target ROI segmentation method based on a priori gating adapter, which can improve the efficiency of segmenting the image corresponding to the infrared weak target in the infrared weak target ROI segmentation process based on a priori gating adapter.

[0023] See Figure 1 As shown, this embodiment of the invention discloses a method for infrared weak target ROI segmentation based on a priori gating adapter, including: Step S11: Input the infrared image into the pre-trained visual basic model encoder embedded with the prior gating adapter, and perform feature extraction under the condition of freezing the backbone parameters of the pre-trained visual basic model to obtain the corresponding multi-scale features.

[0024] In this embodiment, the present application performs infrared weak target ROI segmentation based on a priori gating adapter. While maintaining the backbone capability of the basic model, the task transfer is completed at low cost. Through "full-image boundary correction + local high-resolution refinement", the high recall, low false alarm and fine boundary segmentation of weak targets are finally achieved.

[0025] in, Figure 2This diagram illustrates the infrared ROI segmentation process for weak targets based on a priori gating adapters, where PGLA represents the priori gating adapter, BERR represents the boundary enhancement residual refinement module, and ROI represents the region of interest (salient target region). The overall process in this embodiment includes: inputting the infrared image into the base model encoder for multi-scale feature extraction; obtaining a coarse full-image segmentation result through a global segmentation head; and using a boundary optimization module to refine the boundary enhancement logits of the coarse segmentation. Based on the refined saliency map, the ROI center is selected and patches are cropped, then input into a patch-level fine segmentation network (PatchSegnet) for local magnification and refinement. Finally, the patch prediction results are back-fused according to weights to obtain the final segmentation result.

[0026] In this embodiment, the present application selects to use a base model and a priori gating lightweight adapter for feature extraction: wherein, Figure 3 The flowchart of the InfMAE base model encoder is shown below. First, in this embodiment, the infrared image is input into the pre-trained base model encoder, such as the InfMAE encoder (i.e., a base model designed specifically for infrared modalities), and multi-scale features (i.e., F1, F2, F3, F4) are extracted. F1 and F2 are 2D feature flows from the convolution stage, which come from Encoder layer 1 and Encoder layer 2, respectively; F3 and F4 come from the token feature flows from the Transformer stage.

[0027] Furthermore, in the convolution stage (2D feature flow) and Transformer stage (Token feature flow) of the encoder, this embodiment of the application chooses to insert a priori gated lightweight adapter (PGLA) respectively. It is worth mentioning that the insertion position is after the multi-head attention branch and the MLP branch, and the backbone features are incrementally adjusted by residual method.

[0028] The prior-gated lightweight adapter consists of a low-rank residual branch and a prior-gated branch, with the following output format: ; in, x The input representation can be in the form of a token sequence or a two-dimensional feature map; ⊙ represents element-wise multiplication; s is a learnable scaling factor used to stabilize the model training process. Δ(x) is the residual increment generated by the low-rank transformation (linear dimensionality reduction / upgrading is used for tokens, and 1×1 convolutional dimensionality reduction / upgrading is used for 2D), and its core function is to compensate for the feature defects of the input representation through a small number of parameters. g(x)∈[0,1] are the prior gating coefficients used to selectively enhance effective feature regions and suppress interference from complex backgrounds. They are obtained by fusing three types of prior information: high-frequency contrast (P), edge intensity (E), and background complexity (B).

[0029] It is worth mentioning that the low-rank residual increment Δ(x) can be transformed in two ways according to the different forms of the input representation, respectively adapting to the token sequence and the two-dimensional feature map input. The specific process is as follows: First, regarding the Token input, In this embodiment, normalization is performed first, followed by low-rank projection: ; in, , r is much smaller than C, indicating a low-rank dimension; For nonlinear activation (GELU), it is worth mentioning that the above structure is equivalent to learning an approximately low-rank incremental mapping in the original space, which significantly reduces the number of parameters and computation.

[0030] When the input is a two-dimensional feature map In this embodiment, normalization and 1×1 convolution are used for low-rank transformation: ; Among them, dimensionality reduction convolution is used to compress the channels from C to r, and dimensionality increase convolution is used to restore the channels to C.

[0031] Furthermore, g(x) is the prior gating coefficient, which is obtained by fusing three types of prior information: high-frequency contrast P, edge intensity E, and background complexity B. The calculation expression is shown below: ; Among them, W p W e W β For learnable weights, The sigmoid activation function is used to normalize the gating coefficients to the [0,1] interval. Notably, the high-frequency contrast prior P is obtained by average pooling the input features within a local window to obtain a smoothing component. The difference between the original features and the smoothing component forms a high-frequency residual. This residual is then aggregated through channel-wise convolution, compressed into a single channel by a 1×1 convolution, and nonlinearly normalized. This residual is used to characterize the degree of local texture and contrast abrupt changes. Furthermore, the edge strength prior E is obtained by first projecting multi-channel features into a single-channel intensity map through a 1×1 convolution, then extracting its edge response using the Sobel operator, adjusting it with a 1×1 convolution, and normalizing it. This prior characterizes the gradient strength of structural boundaries. The background complexity prior B is obtained by aggregating the input features along the channel dimension to obtain a single-channel intensity map, calculating its statistical variance within a local window to characterize local undulations and texture clutter, and then performing a 1×1 convolution and normalization. This prior characterizes the intensity of background interference. In this way, the embodiments of this application achieve the enhancement of the effective region and the suppression of complex backgrounds by fusion calculation of the gating coefficients.

[0032] In this embodiment, the present application embodiment achieves low-cost and controllable task transfer by inserting a priori gated lightweight adapter into the key branch of the encoder and freezing the trunk parameters, and only training the adapter and gate weights; at the same time, it utilizes the gate mechanism of "high frequency / edge enhancement and background complexity suppression" to improve the saliency and separability of weak target features and reduce false detection and response fluctuations in complex backgrounds.

[0033] Specifically, the prior gating adapter is embedded in the pre-trained visual base model encoder to generate gating coefficients based on the prior information of contrast, edge intensity, and background complexity of the infrared image, and to perform lightweight fine-tuning of the features in the encoder based on the gating coefficients.

[0034] Step S12: Input the multi-scale features into the global segmentation head for feature fusion and segmentation prediction, and upsample the segmentation results to the original image resolution to obtain the full-image coarse segmentation prediction result.

[0035] In this embodiment, the present application embodiment can generate full-image coarse segmentation logits (denoted as global_up) based on the obtained features at each scale and using a pyramid fusion global segmentation head (using UPerHead), and upsample to the original image resolution to obtain the global position and approximate shape of the target, so as to stably give the global position and initial shape of weak targets, and provide reliable prior guidance and candidate region basis for subsequent boundary refinement and ROI refinement.

[0036] In one specific implementation, after obtaining global_up, this application introduces a Boundary-Enhanced Residual Refiner (BERR) module to incrementally correct the logits of the entire image. BERR is used to first refine the input image... x The grayscale intensity map gray(x) is calculated, and the edge map edge(x) is obtained by using the Sobel operator. Then, the three are concatenated along the channel dimension to obtain the final image. u This is done to achieve feature fusion. Finally, the boundary enhancement logits (denoted as global_ref) are obtained by correcting the boundary residual Δ, and the corresponding expression is shown below: ; ; in, It is a fusion network composed of multiple convolutions, normalization, and ReLU.

[0037] Subsequently, this embodiment of the application requires foreground channel extraction to obtain a refined saliency map (denoted as global_ref_fg), providing high-quality guidance for subsequent ROI selection. In this way, this embodiment of the application uses residual boundary correction on the full image logits by introducing image intensity and edge information, making the segmentation contour more closely match the real edge gradient, thereby significantly improving boundary segmentation accuracy while maintaining global semantic consistency.

[0038] Step S13: Generate a grayscale intensity map based on the infrared image, and calculate a first edge map based on the grayscale intensity map using an edge detection operator; stitch the full image coarse segmentation prediction result, the grayscale intensity map, and the first edge map together in the channel dimension, and input the resulting stitched feature map into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result.

[0039] In this embodiment, the corresponding probability map can be determined based on the boundary enhancement segmentation result global_ref_fg: ; in, This is the sigmoid function.

[0040] Specifically, the coarse segmentation prediction result of the entire image, the grayscale intensity map, and the first edge map are concatenated along the channel dimension. The resulting concatenated feature map is then input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result. This process can include: concatenating the coarse segmentation prediction result of the entire image, the grayscale intensity map, and the first edge map along the channel dimension to obtain the concatenated feature map; inputting the concatenated feature map into the boundary enhancement residual refinement module into a residual network composed of multiple convolutions, normalization, and nonlinear activations to predict the residual terms corresponding to the concatenated feature map; and correcting the coarse segmentation prediction result of the entire image based on the residual terms to obtain the boundary enhancement segmentation result.

[0041] Step S14: Generate a saliency probability map based on the boundary enhancement segmentation result, and perform threshold filtering and pooling operations on the saliency probability map to determine the candidate Region of Interest (ROI); based on the ROI, crop the infrared image, the saliency probability map, and the corresponding edge map to obtain image patches, guide probability patches, and edge patches.

[0042] In this embodiment, pooling and thresholding are performed at a preset step size (e.g., stride=16) to obtain a set of candidate ROI centers. When there are too many candidates, the TopK strategy is used to retain the K most significant centers to avoid invalid computation. Subsequently, based on each ROI center, fixed-size image patches are cropped from the original image, and corresponding guided logits patches (denoted as base_logits) are cropped from global_ref_fg. The patches are then upsampled to a uniform resolution to prepare for subsequent fine segmentation, thereby achieving "target-centered" sparse computation. This concentrates fine processing on potential targets and uncertain regions, improving the recall capability of weak targets and reducing the computational overhead of invalid backgrounds.

[0043] Furthermore, in this embodiment, each ROI is constructed as a patch-level refined input, and each ROI consists of an image patch (base_image) and a guiding prior patch. The guiding prior includes base_prob and an edge map base_edge, forming the input [base_image, base_prob, base_edge]. Subsequently, the above input is fed into the patch-level fine-grained segmentation network PatchSegnet, thereby generating a refined logits patch for the ROI. ; ; In this embodiment, PatchSegnet is used to generate prediction results with the same resolution as the input, and the corresponding network input is a tensor formed by splicing two three-channel data in the channel dimension: the first channel is the image branch input (copying a single-channel image into a three-channel image); the second channel is the edge enhancement branch input [base_image, base_prob, base_edge], where base_image is a grayscale image.

[0044] Furthermore, PatchSegnet includes a dual-branch coding module, a multi-scale dual-stream fusion module, and a decoding module. The dual-branch coding module consists of two encoders with identical structures and independent parameters: an image encoder and an edge enhancement encoder. Both encoders are based on ResNet18 with structural improvements to extract multi-scale features while preserving spatial resolution as much as possible.

[0045] It is worth mentioning that each encoder sequentially includes a first stage of Stem feature extraction, a second stage of residual feature extraction, and a third stage of residual feature extraction, specifically: The first stage (Stem) adopts a deep Stem structure consisting of three 3×3 convolutional layers, with each convolutional layer having a stride of 1. Batch normalization (BN) and ReLU activation functions are applied sequentially after each convolutional layer. This stage replaces the original ResNet18's 7×7 convolution and max pooling operations to complete the initial feature extraction at the input resolution H×W, while effectively reducing the loss of feature details.

[0046] The second stage (Stage-1 of residual blocks): The output of the first stage is input into the first set of residual blocks (layer 1) of ResNet18. The convolution stride of this stage is set to 1. Its core function is to further mine feature information and improve feature representation ability while keeping the feature space resolution unchanged.

[0047] The third stage (Stage-2 of residual blocks): The output of the second stage is input into the second set of residual blocks (layer 2) of ResNet18. This stage performs the only downsampling operation in the entire encoding process. The downsampling stride is set to 2. By downsampling, a larger feature receptive field is obtained, and a medium-to-low resolution feature map is formed, finally obtaining the third-level features.

[0048] The fourth stage (Residual Block Stage-3): The third set of residual blocks (layer 3) of ResNet18 is taken as the core structure of this stage, and the stride of the first residual unit in the preset number of residual units is set to 1, thereby canceling the downsampling operation of this stage; while maintaining the feature resolution of H / 2×W / 2, this stage further improves the deep feature expression capability, and finally obtains the fourth level feature.

[0049] Each encoder outputs four levels of feature maps, and the specific specifications of each level of feature map are as follows: ① First-level features: 64 channels, spatial resolution of H×W; ② Second-level features: 64 channels, spatial resolution of H×W; ③ Third-level features: 128 channels, spatial resolution of H / 2×W / 2; ④ Fourth-level features: 256 channels, spatial resolution of H / 2×W / 2.

[0050] Furthermore, the multi-scale dual-stream fusion module sets up fusion units at the feature output end of each scale. The core function is to fuse the two branch features at the same scale. Each fusion unit first concatenates the image branch feature i at the corresponding scale with the image boundary enhancement branch feature along the channel dimension. Then, it is processed by 3×3 convolution, batch normalization (BN), and ReLU activation function to complete channel compression and deep mixing of feature information, finally obtaining four-level fused features (f1, f2, f3, f4).

[0051] Specifically, a saliency probability map is generated based on the boundary enhancement segmentation results, and thresholding and pooling operations are performed on the saliency probability map to determine candidate Regions of Interest (ROIs). Based on the ROIs, the infrared image, the saliency probability map, and the corresponding edge map are cropped to obtain image patches, guiding probability patches, and edge patches. This can include: mapping the boundary enhancement segmentation results using a sigmoid function to obtain the saliency probability map; pooling the saliency probability map according to a preset step size to obtain a probability response map; thresholding the probability response map and determining the pixel positions with response values ​​greater than a preset threshold as candidate ROI centers; when the number of candidate ROI centers is greater than a preset number, the candidate ROI centers are sorted from high to low according to the response value to select a preset number of ROI centers from the sorted results; a fixed-size ROI region is generated based on each ROI center, and image patches and probability patches are cropped based on the ROI region, along with the infrared image, the saliency probability map, and the corresponding edge map, to obtain image patches, guiding probability patches, and edge patches.

[0052] Furthermore, the process of obtaining edge patches may include: performing an S-shaped function transformation on the guided probability patch to obtain a probability map; processing the probability map using an edge detection operator to obtain a corresponding second edge map, and generating edge patches based on the ROI and the second edge map.

[0053] Step S15: The image patch, the guiding probability patch, and the edge patch are spliced ​​together in the channel dimension, and the splicing result is upsampled and then input into the patch-level fine segmentation network to obtain the corresponding segmentation prediction patch.

[0054] In this embodiment, the decoding module adopts a lightweight structure design. Its core function is to restore the low-resolution fused features step by step to the same H×W resolution as the network input. Its specific structure includes three core units, and the detailed design is shown below: Bottom fusion unit: The fourth-level fusion feature f4 and the third-level fusion feature f3 are concatenated along the channel dimension. After concatenation, they are processed by 3×3 convolution, batch normalization (BN) and ReLU activation function to obtain the intermediate feature d3. The intermediate feature d3 has 128 channels and a spatial resolution of H / 2×W / 2.

[0055] Upsampling fusion unit: First, a 1×1 convolution operation is performed on the intermediate feature d3 to expand its number of channels to 4C (where C is the original number of channels of d3). Then, a 2x upsampling operation is performed through PixelShuffle to restore the feature spatial resolution to H×W. After upsampling, it is concatenated with the second-level fusion feature f2 of the same resolution by channel dimension. The concatenated feature is then processed by 3×3 convolution, batch normalization (BN), and ReLU activation function. At the same time, a coordinate attention module (CoordAtt) is introduced to enhance the feature, and finally feature d2 is obtained. The number of channels of feature d2 is 64 and the spatial resolution is H×W.

[0056] The pre-output fusion unit concatenates feature d2 with the first-level fusion feature f1 along the channel dimension. The concatenated feature is then processed by 3×3 convolution, batch normalization (BN), and ReLU activation function. Finally, a coordinate attention module (CoordAtt) is introduced to complete the final feature optimization, resulting in the final decoded feature d1. The final decoded feature d1 has 64 channels and a spatial resolution of H×W.

[0057] In this way, the embodiments of this application use "image patch + coarse segmentation probability and edge prior" as input to guide the probability patch network to correct and refine local areas, focusing on repairing the fine structure and boundary details of weak targets, effectively improving the problems of adhesion and rough contour, and thus achieving pixel-level fine segmentation.

[0058] Specifically, the patch-level fine segmentation network includes a dual-branch coding module, a multi-scale dual-stream fusion module, and a decoding module. The dual-branch coding module comprises an independent image coding branch and a guiding coding branch. The image coding branch extracts image patch features, and its input is a three-channel image patch formed by copying a single-channel image patch. The guiding coding branch extracts guiding patch features, and its input is a three-channel guiding patch formed by concatenating a single-channel image patch, a segmentation patch, and an edge patch along the channel dimension. The multi-scale dual-stream fusion module fuses the image patch features and guiding patch features at different scales. The decoding module performs progressive upsampling and feature reconstruction on the fused features, outputting a segmentation prediction patch.

[0059] Step S16: Based on the ROI position corresponding to each segmentation prediction patch, map the segmentation prediction patch back to the original image coordinate system, and perform weighted fusion on the overlapping areas in each mapping result to obtain the infrared weak target segmentation result.

[0060] In this embodiment, all ROI fine prediction patches are back-pasted to the original image coordinate system according to their center positions, and the overlapping areas are weighted and fused using distance weights (such as Gaussian weights) to obtain full-image fine segmentation logits. Furthermore, for areas not covered by ROIs, the global_ref_fg result is retained as background / non-key region, and the final full-image segmentation result is output. In this way, this embodiment seamlessly integrates the patch refinement results back into the full image through weighted back-pasting fusion, suppressing patch stitching seams and block artifacts, ensuring the continuity and consistency of full-image prediction, and thus obtaining higher accuracy and more stable final segmentation results with lower computational cost.

[0061] Specifically, based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in the mapping results are weighted and fused to obtain the infrared weak target segmentation result. This can include: mapping the segmentation prediction patch to the original image coordinate system according to the ROI position corresponding to each segmentation prediction patch to obtain several mapping results; when there are overlapping areas in the mapping results, the predicted values ​​at the same pixel position are weighted and fused according to preset weights to obtain a weighted fusion result; when there are no overlapping areas in the mapping results, the predicted value of the corresponding patch is determined; and the infrared weak target segmentation result is generated based on the weighted fusion result and the predicted value.

[0062] As can be seen from the above, the embodiments of this application first require inputting the infrared image into a pre-trained visual base model encoder embedded with a priori gating adapter. Feature extraction is performed under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. Secondly, the multi-scale features are input into a global segmentation head for feature fusion and segmentation prediction, and the segmentation result is upsampled to the original image resolution to obtain a coarse full-image segmentation prediction result. A grayscale intensity map is generated based on the infrared image, and a first edge map is calculated based on the grayscale intensity map using an edge detection operator. Then, the coarse full-image segmentation prediction result, the grayscale intensity map, and the first edge map are stitched together along the channel dimension, and the resulting stitched feature map is input into a boundary enhancement residual refinement module for residual correction to obtain... The process involves several steps: First, boundary enhancement segmentation results are used. Then, a saliency probability map is generated based on the boundary enhancement segmentation results, and thresholding and pooling operations are performed on the saliency probability map to determine candidate Regions of Interest (ROIs). Based on the ROIs, the infrared image, saliency probability map, and corresponding edge map are cropped to obtain image patches, guiding probability patches, and edge patches. Next, the image patches, guiding probability patches, and edge patches are concatenated along the channel dimension, and the concatenated result is upsampled and input into a patch-level fine-grained segmentation network to obtain corresponding segmentation prediction patches. Finally, based on the ROI positions corresponding to each segmentation prediction patch, the segmentation prediction patches are mapped back to the original image coordinate system, and overlapping areas in the mapping results are weighted and fused to obtain the infrared weak target segmentation result. This approach improves the segmentation accuracy of infrared weak targets in complex backgrounds and enhances the target boundary representation effect during the infrared weak target ROI segmentation process based on a priori gating adapter, thereby improving the recognition capability of low signal-to-noise ratio and small-scale targets.

[0063] Accordingly, see Figure 4 As shown, this application also provides an infrared ROI segmentation device for small targets based on a priori gating adapter, comprising: The multi-scale feature generation module 11 is used to input the infrared image into the pre-trained visual base model encoder embedded with the prior gating adapter, and to extract features under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. The full-image coarse segmentation prediction result generation module 12 is used to input the multi-scale features into the global segmentation head for feature fusion and segmentation prediction, and upsample the segmentation result to the original image resolution to obtain the full-image coarse segmentation prediction result. The boundary enhancement segmentation result generation module 13 is used to generate a grayscale intensity map based on the infrared image, and calculate a first edge map based on the grayscale intensity map using an edge detection operator; the whole image coarse segmentation prediction result, the grayscale intensity map and the first edge map are stitched together in the channel dimension, and the resulting stitched feature map is input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result; The ROI determination module 14 is used to generate a saliency probability map based on the boundary enhancement segmentation result, and perform threshold filtering and pooling operations on the saliency probability map to determine the ROI of the candidate region of interest; and to crop the infrared image, the saliency probability map and the corresponding edge map based on the ROI to obtain image patches, guide probability patches and edge patches. The segmentation prediction patch generation module 15 is used to stitch the image patch, the guiding probability patch and the edge patch in the channel dimension, and input the stitching result into the patch-level fine segmentation network after upsampling to obtain the corresponding segmentation prediction patch. The infrared weak target segmentation result generation module 16 is used to map the segmentation prediction patch back to the original image coordinate system based on the ROI position corresponding to each segmentation prediction patch, and to perform weighted fusion on the overlapping areas in each mapping result to obtain the infrared weak target segmentation result.

[0064] In some specific embodiments, the boundary enhancement segmentation result generation module 13 may specifically include: The feature map construction unit is used to stitch together the coarse segmentation prediction result of the whole image, the gray intensity map and the first edge map in the channel dimension to obtain the stitched feature map; The residual term generation unit is used to input the stitched feature map into the residual network composed of multiple convolutions, normalization and nonlinear activation in the boundary enhancement residual refinement module, so as to use the residual network to predict the residual term corresponding to the stitched feature map. The prediction result correction unit is used to correct the coarse segmentation prediction result of the whole image based on the residual term to obtain the boundary enhancement segmentation result.

[0065] In some specific embodiments, the ROI determination module 14 may specifically include: A saliency probability map generation unit is used to map the boundary enhancement segmentation result through a sigmoid function to obtain a saliency probability map; The probability response map determination unit is used to perform pooling processing on the saliency probability map according to a preset step size to obtain the probability response map; The candidate ROI center determination unit is used to perform threshold screening on the probability response map and determine the pixel positions with response values ​​greater than a preset threshold as candidate ROI centers. The candidate ROI center sorting unit is used to sort the candidate ROI centers from high to low according to the response value when the number of candidate ROI centers is greater than a preset number, so as to select a preset number of ROI centers from the sorting results. The patch generation unit is used to generate a fixed-size ROI region based on the center of each ROI, and to perform image patching and probability patching on the infrared image, the saliency probability map and the corresponding edge map based on the ROI region to obtain image patch, guide probability patch and edge patch.

[0066] In some specific embodiments, the ROI determination module 14 may specifically include: The unit for determining the center of the ROI to be processed is used to perform an S-shaped function transformation on the guiding probability patch to obtain a probability map. An edge map generation unit is used to process the probability map using an edge detection operator to obtain a corresponding second edge map, so as to generate an edge patch based on the ROI and the second edge map.

[0067] In some specific embodiments, the infrared weak target segmentation result generation module 16 may specifically include: The mapping result generation unit is used to map the segmentation prediction patch to the original image coordinate system according to the ROI position corresponding to each segmentation prediction patch, and obtain several mapping results; The weighted fusion result generation unit is used to perform weighted fusion on the predicted values ​​of the same pixel position according to a preset weight when there are overlapping areas in the mapping results, so as to obtain a weighted fusion result; The prediction value determination unit is used to determine the prediction value of the corresponding patch when there is no overlapping region in each of the mapping results; The infrared weak target segmentation result generation subunit is used to generate infrared weak target segmentation results based on the weighted fusion result and the predicted value.

[0068] Furthermore, embodiments of this application also disclose an electronic device, Figure 5This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the infrared weak target ROI segmentation method based on a priori gating adapter disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0069] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0070] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0071] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the infrared weak target ROI segmentation method based on a priori gating adapter executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0072] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned infrared weak target ROI segmentation method based on a priori gating adapter. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0074] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0075] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0076] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0077] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for infrared ROI segmentation of small targets based on a priori gating adapters, characterized in that, include: Infrared images are input into a pre-trained visual base model encoder embedded with a priori gating adapter. Feature extraction is performed under the condition that the backbone parameters of the pre-trained visual base model are frozen to obtain the corresponding multi-scale features. The multi-scale features are input into the global segmentation head for feature fusion and segmentation prediction, and the segmentation results are upsampled to the original image resolution to obtain the full-image coarse segmentation prediction results. A grayscale intensity map is generated based on the infrared image, and a first edge map is calculated based on the grayscale intensity map using an edge detection operator; The coarse segmentation prediction result of the whole image, the gray intensity map and the first edge map are stitched together in the channel dimension, and the resulting stitched feature map is input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result. A saliency probability map is generated based on the boundary enhancement segmentation result, and threshold filtering and pooling operations are performed on the saliency probability map to determine the candidate Region of Interest (ROI). Based on the ROI, the infrared image, the saliency probability map, and the corresponding edge map are cropped to obtain image patches, guide probability patches, and edge patches. The image patch, the guiding probability patch, and the edge patch are concatenated along the channel dimension, and the concatenation result is upsampled and then input into the patch-level fine segmentation network to obtain the corresponding segmentation prediction patch. Based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in each mapping result are weighted and fused to obtain the infrared weak target segmentation result.

2. The method according to claim 1, characterized in that, The prior gating adapter is embedded in the pre-trained visual base model encoder and is used to generate gating coefficients based on the prior information of contrast, edge intensity, and background complexity of the infrared image, and to perform lightweight fine-tuning of the features in the encoder based on the gating coefficients.

3. The method according to claim 1, characterized in that, The process involves concatenating the full-image coarse segmentation prediction result, the grayscale intensity map, and the first edge map along the channel dimension, and then inputting the resulting concatenated feature map into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result, including: The coarse segmentation prediction result of the whole image, the grayscale intensity map and the first edge map are concatenated in the channel dimension to obtain the concatenated feature map. In the boundary enhancement residual refinement module, the stitched feature map is input into a residual network consisting of multiple convolutions, normalization, and nonlinear activations, so as to use the residual network to predict the residual terms corresponding to the stitched feature map. The coarse segmentation prediction result of the whole image is corrected based on the residual term to obtain the boundary enhancement segmentation result.

4. The method according to claim 1, characterized in that, The process involves generating a saliency probability map based on the boundary enhancement segmentation results, and then performing threshold filtering and pooling operations on the saliency probability map to determine the candidate Region of Interest (ROI). Based on the ROI, the infrared image, the saliency probability map, and the corresponding edge map are cropped to obtain image patches, guidance probability patches, and edge patches, including: The boundary enhancement segmentation results are mapped using a sigmoid function to obtain a saliency probability map; The saliency probability map is pooled according to a preset step size to obtain a probability response map; The probability response map is subjected to threshold filtering, and the pixel positions with response values ​​greater than a preset threshold are determined as candidate ROI centers; When the number of candidate ROI centers is greater than the preset number, the candidate ROI centers are sorted from high to low according to the response value, so as to select a preset number of ROI centers from the sorting results. Based on the centers of each ROI, a fixed-size ROI region is generated, and based on the ROI region, the infrared image, the saliency probability map, and the corresponding edge map are cropped by image patching and probability patching to obtain image patch, guide probability patch, and edge patch.

5. The method according to claim 4, characterized in that, The process of obtaining the edge patch includes: The probability map is obtained by performing an S-shaped function transformation on the guiding probability patch. The probability map is processed using an edge detection operator to obtain a corresponding second edge map, and an edge patch is generated based on the ROI and the second edge map.

6. The method according to claim 1, characterized in that, The patch-level fine segmentation network includes a dual-branch coding module, a multi-scale dual-stream fusion module, and a decoding module; The dual-branch coding module includes an independent image coding branch and a guide coding branch. The image coding branch is used to extract image patch features, and the input of the image coding branch is a three-channel image patch formed by copying a single-channel image patch. The guide coding branch is used to extract guide patch features, and the input of the guide coding branch is a three-channel guide patch formed by concatenating a single-channel image patch, a segmentation patch, and an edge patch in the channel dimension. The multi-scale dual-stream fusion module is used to fuse the image patch features and the guide patch features at different scales; The decoding module is used to perform step-by-step upsampling and feature reconstruction on the fused features and output segmentation prediction patches.

7. The method according to claim 1, characterized in that, Based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped back to the original image coordinate system, and the overlapping areas in each mapping result are weighted and fused to obtain the infrared weak target segmentation result, including: Based on the ROI position corresponding to each segmentation prediction patch, the segmentation prediction patch is mapped to the original image coordinate system to obtain several mapping results; When there are overlapping regions in the mapping results, the predicted values ​​at the same pixel position are weighted and fused according to the preset weights to obtain the weighted fusion result. When there are no overlapping regions in the mapping results, the predicted value of the corresponding patch is determined; Infrared weak target segmentation results are generated based on the weighted fusion result and the predicted value.

8. An infrared ROI segmentation device for small targets based on a priori gating adapter, characterized in that, include: The multi-scale feature generation module is used to input infrared images into a pre-trained visual base model encoder embedded with a priori gating adapter, and to extract features under the condition of freezing the backbone parameters of the pre-trained visual base model to obtain the corresponding multi-scale features. The full-image coarse segmentation prediction result generation module is used to input the multi-scale features into the global segmentation head for feature fusion and segmentation prediction, and upsample the segmentation result to the original image resolution to obtain the full-image coarse segmentation prediction result. The boundary enhancement segmentation result generation module is used to generate a grayscale intensity map based on the infrared image, and to calculate a first edge map based on the grayscale intensity map using an edge detection operator; The coarse segmentation prediction result of the whole image, the gray intensity map and the first edge map are stitched together in the channel dimension, and the resulting stitched feature map is input into the boundary enhancement residual refinement module for residual correction to obtain the boundary enhancement segmentation result. The ROI determination module is used to generate a saliency probability map based on the boundary enhancement segmentation result, and perform threshold filtering and pooling operations on the saliency probability map to determine the ROI of the candidate region of interest; based on the ROI, the infrared image, the saliency probability map and the corresponding edge map are cropped to obtain image patches, guide probability patches and edge patches. The segmentation prediction patch generation module is used to stitch the image patch, the guiding probability patch and the edge patch in the channel dimension, and input the stitching result into the patch-level fine segmentation network after upsampling to obtain the corresponding segmentation prediction patch. The infrared weak target segmentation result generation module is used to map the segmentation prediction patch back to the original image coordinate system based on the ROI position corresponding to each segmentation prediction patch, and to perform weighted fusion on the overlapping areas in each mapping result to obtain the infrared weak target segmentation result.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer programs; The processor is used to execute the computer program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images