Camouflage object semantic segmentation method and device based on adaptive candidate strategy, medium

Abstract

The disclosure provides a camouflage object semantic segmentation method and device based on an adaptive candidate strategy, and a medium. The technical scheme of the disclosure obtains the class code of a to-be-segmented image through a coarse feature extractor and a classifier, realizes the transmission of semantic class information, and enhances the semantic perception ability of a subsequent segmentation task. An initial prediction of a camouflage object is performed through a target detector and an adaptive candidate strategy, and an optimal target attention box is generated to provide robust spatial guidance for a core segmentation task. The class code and the target attention box are combined, multi-source information is fused and reconstructed through a multi-guided feature fusion module and a multi-task perception decoder, and an accurate segmentation result is output.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and image processing technology, and in particular to a method, apparatus, and medium for semantic segmentation of camouflaged objects based on an adaptive candidate strategy. Background Technology

[0002] The camouflaged object semantic segmentation task aims to identify target objects in input images that are visually indistinguishable from their surroundings. Because camouflaged objects are highly similar to the background in color, texture, and shape, this task poses a significant challenge to traditional object detection and segmentation techniques and has driven research into fine-grained visual perception capabilities. This technology has broad applications in various fields, such as polyp segmentation in medical images, surface defect detection in industrial manufacturing processes, and anti-camouflage military reconnaissance and emergency rescue scenarios.

[0003] In existing technologies, semantic segmentation methods for camouflaged objects struggle to fully capture the diversity of camouflaged targets in terms of size, shape, and spatial location, resulting in insufficient segmentation accuracy and robustness. Summary of the Invention

[0004] This disclosure provides at least one method, apparatus, and medium for semantic segmentation of camouflaged objects based on an adaptive candidate strategy, in order to solve at least one of the aforementioned technical problems.

[0005] According to another aspect of this disclosure, a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy is provided, comprising:

[0006] Obtain the image to be segmented;

[0007] The image to be segmented is subjected to multi-scale feature extraction using a coarse feature extractor to obtain multiple classification feature maps at different scales. Then, the multiple classification feature maps at different scales are clustered using a classifier to obtain the category code of the image to be segmented.

[0008] The image to be segmented is encoded and localization features are extracted using an object detector to obtain multiple localization feature maps of different scales. An adaptive candidate strategy is then used to dynamically parse the multiple localization feature maps of different scales to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects.

[0009] The image to be segmented is cropped based on the target attention box to obtain cropped image blocks, and the cropped image blocks are magnified by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks.

[0010] The category encoding and the multi-scale perceptual attention region image patch are input into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. The multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales to obtain fine-grained feature representations of the disguised object at multiple different scales. The multi-task perceptual decoder is used to perform group feature interaction and stepwise fusion on the multiple fine-grained feature representations at different scales, and the segmentation result of the disguised object is determined based on the fusion result.

[0011] In one possible implementation, the coarse feature extractor includes a ResNet backbone network or a Transformer backbone network;

[0012] The ResNet backbone network is based on a residual block stacking structure, and the residual block includes two convolutional layers, a batch normalization layer, and a ReLU activation function;

[0013] The Transformer backbone network includes a PVTv2 network, which includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0014] In one possible implementation, the classifier includes a module for minimizing the intra-class squared distance error and a module for iteratively updating cluster centers;

[0015] The module for minimizing intra-class squared distance error is used to calculate the distance between the input feature vector and the current cluster center based on any of the metrics such as Euclidean distance, cosine similarity, or Mahalanobis distance, and to assign the feature vector to the cluster to which the cluster center with the smallest distance belongs, thereby minimizing the intra-class sum of squares error.

[0016] The iterative cluster center update module is used to recalculate the mean of all feature vectors in each cluster after completing a feature vector classification, and use the new mean as the new cluster center.

[0017] In one possible implementation, the target detector includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network, which includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0018] In one possible implementation, the module that performs the adaptive candidate strategy includes a prediction tensor parsing branch, a confidence filtering branch, a clipping box generation branch, and a boundary compensation branch.

[0019] The prediction tensor parsing branch is used to perceive the spatial and semantic information of potential camouflaged objects from the input localization feature map. The prediction tensor is obtained through an end-to-end detection model. The prediction tensor includes the position information of the corresponding potential camouflaged object and the predicted category. The position information includes the center position offset, width and height information, and confidence score of the potential camouflaged object. By traversing the grid cells of the localization feature map, the center position offset in the prediction tensor is normalized and mapped to the space of the image to be segmented, and the center coordinates of the candidate target box are obtained by combining the width and height information.

[0020] The confidence filtering branch is used to select the candidate target box with the highest confidence and a confidence level higher than a preset threshold as the final candidate box; if the confidence level of all candidate target boxes is lower than the preset threshold, then the box corresponding to the entire localization feature map is used as the final candidate box.

[0021] The clipping box generation branch is used to construct a clipping box that meets the side length constraints based on the center coordinates of the final candidate box;

[0022] The boundary compensation branch is used to calculate the position information of the out-of-bounds portion when the coordinates of the cropping box exceed the boundary of the image to be segmented, and to perform translation compensation based on the position information so that the cropping box falls back into the effective area of ​​the image to be segmented while maintaining its original size, thus obtaining the target attention box.

[0023] In one possible implementation, the backbone network includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network; the PVTv2 network includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0024] In one possible implementation, the multi-guided feature fusion module includes a feature alignment and interaction submodule and a feature reconstruction and enhancement submodule;

[0025] The feature alignment and interaction submodule includes a feature preprocessing branch, a spatial alignment branch, and a multi-head attention branch;

[0026] The feature preprocessing branch is used to normalize semantic feature maps of multiple different scales and apply convolution operations with kernels of a preset size to achieve consistency of feature distribution and suppression of redundant information.

[0027] The spatial alignment branch is used to process the features output by the feature preprocessing branch through three parallel max pooling layers and a global average pooling layer to generate spatial description information at different scales; and, by using convolutional layers, normalization layers and ReLU activation functions in sequence to process the spatial description information at different scales, the spatial resolution is adaptively adjusted and the scale features are aligned.

[0028] The multi-head attention branch is used to process the features output by the spatial alignment branch using a multi-head self-attention mechanism to capture long-range dependencies and cross-scale feature interaction information, and to obtain a stable feature representation with enhanced features by combining a normalization layer and skip connections.

[0029] The feature reconstruction and enhancement submodule includes a multi-receptive-field convolutional branch and a gated activation branch;

[0030] The multi-receptive-field convolutional branch is used to process the stable feature representation through a deep convolutional layer with three different convolutional kernels to obtain a joint feature representation of global and local semantic information.

[0031] The gated activation branch is used to process the joint feature representation of global and local semantic information through convolution, normalization layer, GeLU activation and concatenation operations, so as to realize dynamic control of the importance of feature channels and spatial location, as well as information enhancement and noise suppression in the fusion process, and obtain fine-grained feature representations of camouflaged objects at multiple different scales.

[0032] In one possible implementation, the multi-task-aware decoder includes a group interaction submodule and a progressive fusion submodule;

[0033] The grouping interaction submodule includes a feature expansion unit, a feature grouping unit, and an intra-group interaction unit;

[0034] The feature unrolling unit is used to process multiple fine-grained feature representations of different scales of input sequentially through a multi-hole convolutional layer, and divide the processing results into several subsets according to channels;

[0035] The feature grouping unit is used to process each subset one by one. The input of the current subset is concatenated with the output of the previous group and then passed through convolution, normalization layer and ReLU activation in sequence to obtain the corresponding in-group output features.

[0036] The intra-group interaction unit is used to perform internal correlation modeling on the intra-group output features to obtain explicit features;

[0037] The progressive fusion submodule includes: a feature reconstruction unit, a fusion generation unit, and a residual output unit;

[0038] The feature reconstruction unit is used to divide the explicit features obtained by the intra-group interaction unit into two subsets, which are then input into the multi-head self-attention mechanism to model cross-regional dependencies and obtain two sets of enhanced features.

[0039] The fusion generation unit utilizes sequentially connected convolutional kernels as... The convolutional layers, normalization layers, and ReLU activation function are used to process the two sets of enhanced features to obtain fused features;

[0040] The residual output unit is used for skip connections and convolution kernels. The convolutional layers, normalization layers, and ReLU activation functions process the fused features to obtain the final fused features; the skip connections are used to maintain the continuity between the original input information of the multi-task perceptual decoder and the final fused features.

[0041] Another aspect of this disclosure provides a semantic segmentation apparatus for camouflaged objects based on an adaptive candidate strategy, comprising:

[0042] The image acquisition module is used to acquire the image to be segmented;

[0043] The initial classification module is used to extract multi-scale features from the image to be segmented using a coarse feature extractor to obtain multiple classification feature maps at different scales, and to cluster the multiple classification feature maps at different scales using a classifier to obtain the category code of the image to be segmented.

[0044] The target determination module is used to encode and extract localization features of the image to be segmented using a target detector to obtain multiple localization feature maps of different scales, and to dynamically parse the multiple localization feature maps of different scales using an adaptive candidate strategy to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects.

[0045] The image enhancement module is used to crop the image to be segmented based on the target attention box to obtain cropped image blocks, and to magnify the cropped image blocks by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks.

[0046] A deep classification module is used to input the category encoding and the multi-scale perceptual attention region image patch into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. A multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales to obtain fine-grained feature representations of the disguised object at multiple different scales. A multi-task perceptual decoder is used to perform group feature interaction and stepwise fusion on the multiple fine-grained feature representations at different scales, and the segmentation result of the disguised object is determined based on the fusion result.

[0047] Another aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the camouflaged object semantic segmentation method based on an adaptive candidate strategy as described above.

[0048] This disclosure discloses a semantic segmentation method, apparatus, and medium for camouflaged objects based on an adaptive candidate strategy. First, a coarse feature extractor is used to extract multi-scale features from the image to be segmented, obtaining multiple classification feature maps at different scales. Then, a classifier is used to cluster these multiple classification feature maps to obtain the category encoding of the image to be segmented. Next, an object detector is used to encode and extract localization features from the image to be segmented, obtaining multiple localization feature maps at different scales. An adaptive candidate strategy is then used to dynamically parse these localization feature maps to determine candidate bounding boxes and target attention boxes corresponding to the camouflaged objects. Finally, the image to be segmented is cropped based on the target attention boxes. The process involves cropping image patches and then magnifying them at multiple magnifications to obtain a set of multi-scale perceptual attention region image patches. Finally, the category encoding and the multi-scale perceptual attention region image patches are input into a backbone network. The backbone network extracts multi-scale semantic features from the input information, resulting in multiple semantic feature maps at different scales. A multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales, yielding fine-grained feature representations of the disguised object at multiple different scales. A multi-task perceptual decoder is used to perform grouped feature interaction and progressive fusion of these fine-grained feature representations at multiple different scales, and the segmentation result of the disguised object is determined based on the fusion result. The technical solution disclosed herein obtains the category encoding of the image to be segmented through a coarse feature extractor and a classifier, thereby conveying semantic category information and enhancing the semantic perception capability of subsequent segmentation tasks. It uses an object detector and an adaptive candidate strategy to perform initial prediction of camouflaged objects and generate optimal target attention boxes, providing robust spatial guidance for the core segmentation task. Combining the category encoding and target attention boxes, a multi-guided feature fusion module and a multi-task perceptual decoder fuse and reconstruct multi-source information, outputting accurate segmentation results. This technical solution establishes a collaborative optimization and progressive modeling relationship among the three stages: a category encoding stage for the classification task, an adaptive target attention box stage for the detection task, and a semantic reconstruction stage based on multi-source features for the segmentation task. This achieves effective fusion between multiple tasks and improves the accurate segmentation capability of camouflaged targets in complex scenes.

[0049] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0050] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0051] Figure 1This is a flowchart of a camouflaged object semantic segmentation method based on an adaptive candidate strategy in one embodiment of this disclosure;

[0052] Figure 2A This is a model structure diagram corresponding to the image classification stage in a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy in one embodiment of the present disclosure.

[0053] Figure 2B This is a model structure diagram corresponding to the image detection stage in a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy in one embodiment of this disclosure.

[0054] Figure 2C This invention provides a model structure diagram for the image segmentation stage and category encoding in a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy in one embodiment of the present disclosure.

[0055] Figure 2D This is a model structure diagram of the multi-guided feature fusion module in a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy in one embodiment of this disclosure.

[0056] Figure 2E This is a model structure diagram of the multi-task perception decoder in a camouflaged object semantic segmentation method based on an adaptive candidate strategy in one embodiment of this disclosure.

[0057] Figure 2F This is a schematic diagram of a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy in one embodiment of this disclosure;

[0058] Figure 3 This is the image to be segmented input in one embodiment of this disclosure;

[0059] Figure 4 This is a schematic diagram of a pixel-level label in one embodiment of the present disclosure;

[0060] Figure 5 This is a rendering of the technical solution disclosed in this publication;

[0061] Figure 6 This is a rendering of CamoDiffusion.

[0062] Figure 7 Here is a screenshot of the VSCode screenshot;

[0063] Figure 8 This is a screenshot of ZoomNeXt.

[0064] Figure 9 This is a schematic diagram of the structure of a camouflaged object semantic segmentation device based on an adaptive candidate strategy in one embodiment of this disclosure. Detailed Implementation

[0065] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0066] This disclosure addresses the shortcomings of poor robustness and low segmentation accuracy in current camouflage object segmentation methods by providing a semantic segmentation method, apparatus, and medium for camouflage objects based on an adaptive candidate strategy. The technical solution of this disclosure provides a category encoding stage for classification tasks, an adaptive target attention box stage for detection tasks, and a semantic reconstruction stage based on multi-source features for segmentation tasks. These three stages form a collaborative optimization and progressive modeling relationship, thereby achieving effective fusion between multiple tasks and improving the ability to accurately segment camouflage targets in complex scenes.

[0067] The technical solution of this disclosure will be described below through specific embodiments.

[0068] like Figure 1 The diagram shows a flowchart of a semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to this embodiment. The execution subject of this embodiment is a computing device, component, or system with data processing capabilities. Specifically, the method of this embodiment may include the following steps:

[0069] S110. Obtain the image to be segmented.

[0070] S120. Use a coarse feature extractor to extract multi-scale features from the image to be segmented to obtain multiple classification feature maps of different scales, and use a classifier to cluster the multiple classification feature maps of different scales to obtain the category code of the image to be segmented.

[0071] The coarse feature extractor includes a ResNet backbone network or a Transformer backbone network; the ResNet backbone network is based on a residual block stacking structure, and the residual block includes two convolutional layers, a batch normalization layer, and a ReLU activation function; the Transformer backbone network includes a PVTv2 network, and the PVTv2 network includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0072] The classifier includes a module for minimizing the intra-class squared distance error and a module for iteratively updating cluster centers. The module for minimizing the intra-class squared distance error is used to calculate the distance between the input feature vector and the current cluster center based on any of the metrics such as Euclidean distance, cosine similarity, or Mahalanobis distance, and to assign the feature vector to the cluster to which the cluster center with the smallest distance belongs, thereby minimizing the intra-class sum of squares error. The module for iteratively updating cluster centers is used to recalculate the mean of all feature vectors in each cluster after completing a feature vector classification, and to use the new mean as the new cluster center.

[0073] This step is the image classification stage. A coarse feature extractor is used to perform preliminary feature modeling on the input image to be segmented, and a K-Means clusterer is used. That is, the classifier clusters the features in an unsupervised manner to obtain the category code corresponding to the image to be segmented, so as to convey semantic category information and enhance the semantic perception ability of subsequent segmentation tasks.

[0074] S130. The image to be segmented is encoded and localization features are extracted using an object detector to obtain multiple localization feature maps of different scales. An adaptive candidate strategy is used to dynamically analyze the multiple localization feature maps of different scales to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects.

[0075] The target detector includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network, which includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0076] The module that executes the adaptive candidate strategy includes a prediction tensor parsing branch, a confidence filtering branch, a clipping box generation branch, and a boundary compensation branch.

[0077] The prediction tensor parsing branch is used to perceive the spatial and semantic information of potential camouflaged objects from the input localization feature map. A prediction tensor is obtained through an end-to-end detection model. The prediction tensor includes the location information of the corresponding potential camouflaged object and the predicted category. The location information includes the center offset, width and height information, and confidence score of the potential camouflaged object. By traversing the grid cells of the localization feature map, the center offset in the prediction tensor is normalized and mapped to the space of the image to be segmented, and the center coordinates of the candidate target boxes are obtained by combining the width and height information. The confidence score filtering branch is used to select the candidate target boxes with the highest confidence score and a confidence score higher than a preset threshold as the final candidate boxes. If the confidence scores of all candidate target boxes are lower than the preset threshold, the boxes corresponding to the entire localization feature map are used as the final candidate boxes. The cropping box generation branch is used to construct cropping boxes that meet the side length constraints based on the center coordinates of the final candidate boxes. The boundary compensation branch is used to calculate the position information of the out-of-bounds portion when the coordinates of the cropping box exceed the boundary of the image to be segmented, and to perform translation compensation based on the position information so that the cropping box falls back into the effective area of ​​the image to be segmented while maintaining its original size, thus obtaining the target attention box.

[0078] This step is the image detection stage. An object detector is used to generate initial predictions, and an optimal object attention box is generated based on an adaptive strategy of confidence filtering and boundary compensation, providing robust spatial guidance for the core segmentation task.

[0079] S140. Based on the target attention box, the image to be segmented is cropped to obtain cropped image blocks, and the cropped image blocks are magnified by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks.

[0080] S150. The category encoding and the multi-scale perceptual attention region image block are input into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. The multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales to obtain multiple fine-grained feature representations of the disguised object at different scales. The multi-task perceptual decoder is used to perform group feature interaction and stepwise fusion on the multiple fine-grained feature representations at different scales, and the segmentation result of the disguised object is determined based on the fusion result.

[0081] The backbone network includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network; the PVTv2 network includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

[0082] The multi-guided feature fusion module includes a feature alignment and interaction submodule and a feature reconstruction and enhancement submodule. The feature alignment and interaction submodule includes a feature preprocessing branch, a spatial alignment branch, and a multi-head attention branch.

[0083] The feature preprocessing branch is used to normalize semantic feature maps at multiple different scales and apply convolution operations with kernels of a preset size to achieve consistent feature distribution and suppress redundant information. The spatial alignment branch is used to process the features output by the feature preprocessing branch through three parallel max pooling layers and a global average pooling layer to generate spatial description information at different scales. It also uses convolutional layers, normalization layers, and ReLU activation functions to process spatial description information at different scales in sequence to achieve adaptive adjustment of spatial resolution and scale feature alignment. The multi-head attention branch is used to process the features output by the spatial alignment branch using a multi-head self-attention mechanism to capture long-range dependencies and cross-scale feature interaction information, and combines normalization layers and skip connections to obtain a stable feature representation with enhanced features.

[0084] The feature reconstruction and enhancement submodule includes a multi-receptive-field convolutional branch and a gated activation branch. The multi-receptive-field convolutional branch processes the stable feature representation through a deep convolutional layer with three different kernels to obtain a joint feature representation of global and local semantic information. The gated activation branch processes the joint feature representation of global and local semantic information through convolution, normalization layers, GeLU activation, and concatenation operations to achieve dynamic control of the importance of feature channels and spatial locations, as well as information enhancement and noise suppression during the fusion process, resulting in fine-grained feature representations of the camouflaged object at multiple different scales.

[0085] The multi-task perceptual decoder includes a group interaction submodule and a progressive fusion submodule. The group interaction submodule includes a feature expansion unit, a feature grouping unit, and an intra-group interaction unit. The feature expansion unit processes multiple fine-grained feature representations of different scales of the input sequentially through multi-dilated convolutional layers, and divides the processing results into several subsets according to channels. The feature grouping unit processes each subset group by concatenating the input of the current subset with the output of the previous group, and then sequentially passing the convolution, normalization, and ReLU activation layers to obtain the corresponding intra-group output features. The intra-group interaction unit models the internal correlations of the intra-group output features to obtain explicit features.

[0086] The progressive fusion submodule includes a feature reconstruction unit, a fusion generation unit, and a residual output unit. The feature reconstruction unit divides the explicit features obtained from the intra-group interaction unit into two subsets, which are then input into a multi-head self-attention mechanism to model cross-regional dependencies and obtain two sets of enhanced features. The fusion generation unit utilizes sequentially connected convolutional kernels... The convolutional layers, normalization layers, and ReLU activation functions process the two sets of enhanced features to obtain fused features; the residual output unit is used to pass skip connections and convolutional kernels... The convolutional layers, normalization layers, and ReLU activation function are used to process the fused features to obtain the final fused features; the skip connections are used to maintain the continuity between the original input information and the final fused features.

[0087] Steps S140-S150 belong to the image segmentation stage. They combine the category codes generated in the classification stage with the target attention boxes obtained in the detection stage. Through a multi-guided feature fusion module and a multi-task perceptual decoder, multi-source information is fused and reconstructed to output accurate segmentation results. This embodiment can effectively achieve fine segmentation of camouflaged objects, significantly improving the accuracy and completeness of detection.

[0088] In some embodiments, step S120 may specifically include: using a coarse feature extractor to encode the input image to be segmented, obtaining a first classification feature map, a second classification feature map, a third classification feature map, and a fourth classification feature map; based on the multi-level classification feature maps, using a K-Means classifier to encode the image into categories, obtaining category codes to introduce prior information about the target's categories. The category codes are injected as additional information into the segmentation network in S150, enhancing the network's category sensitivity during unsupervised training, thereby improving its ability to distinguish subtle differences between different camouflaged targets. The aforementioned classification feature maps are multi-scale feature representations extracted from different network layers, corresponding to the network's representational ability of the input image to be segmented at different depths. Their difference lies in that: low-level feature maps (such as the first classification feature map) mainly reflect fine-grained structural information such as edges and textures; high-level feature maps (such as the fourth classification feature map) contain more semantic-level or category-discrimination information.

[0089] A coarse feature extractor is used to encode and extract features from the image to be segmented input in the classification stage to obtain classification features for camouflaged objects; a K-Means classifier is used to perform unsupervised clustering of camouflaged objects based on the classification features and generate category codes.

[0090] The coarse feature extractor is composed of a ResNet backbone network or a Transformer backbone network. The ResNet backbone network includes, but is not limited to, any of the following implementations: a ResNet50 network based on a stacked residual block structure, using basic residual units composed of two layers of convolution, batch normalization, and ReLU activation functions to implement feature modeling, or its equivalent or improved structures. The Transformer backbone network includes, but is not limited to, any of the following implementations: a PVTv2 network based on a multi-head self-attention mechanism, a feedforward neural network, layer normalization, and residual connections, or its equivalent or improved structures.

[0091] The K-Means classifier includes a module for minimizing intra-class squared distance error and a module for iteratively updating cluster centers. The module for minimizing intra-class squared distance error calculates the distance between the input feature vector and the current cluster center based on any one of Euclidean distance, cosine similarity, or Mahalanobis distance, and assigns the feature vector to the cluster with the smallest distance, thereby minimizing the intra-class sum of squares error. The module for iteratively updating cluster centers recalculates the mean of all feature vectors within each cluster after each sample allocation, using this as the new cluster center, and repeats this update process until a preset condition or the maximum number of iterations is reached.

[0092] In some embodiments, step S130 may specifically include: using a target detector to perform preliminary localization of the target region in the input image to be detected, i.e., the region where the disguised object is located, and obtaining a first localization feature map, a second localization feature map, a third localization feature map, and a fourth localization feature map respectively; based on the multi-level localization feature maps, using an adaptive candidate strategy to dynamically parse uncertain regions and generate target attention boxes; the adaptive candidate strategy can adaptively focus on key regions, providing accurate spatial prior guidance for subsequent segmentation tasks.

[0093] The first to fourth localization feature maps are multi-scale feature representations output by the target detector (Transformer backbone, such as PVTv2) at different levels, used to capture spatial location information of different granularities. Similarly, the shallower the level, the higher the spatial resolution and the richer the details and boundary information; the deeper the level, the more suitable it is for overall target localization and semantic reasoning.

[0094] The object detector is used to encode and extract features from the input image to be segmented in order to obtain the localization features of the disguised object; the adaptive candidate strategy constructs a "perception-decision-execution" closed-loop mechanism based on the localization features, first perceiving the uncertain region, and then generating the optimal target attention box through adaptive logic, which serves as a strong spatial prior to guide the segmentation.

[0095] The target detector is composed of a Transformer backbone network; the Transformer backbone network includes, but is not limited to, any of the following implementations: a PVTv2 network based on multi-head self-attention mechanism, feedforward neural network, layer normalization and residual connection, or its equivalent or improved structure.

[0096] The module that executes the adaptive candidate strategy includes a prediction tensor parsing branch, a confidence screening branch, a clipping box generation branch, and a boundary compensation branch. These four branches work together to build a closed-loop mechanism of "perception-decision-execution".

[0097] The prediction tensor parsing branch is responsible for perceiving the spatial and semantic information of potential targets from the input image to be segmented, and obtaining prediction tensors through an end-to-end detection model. Each prediction tensor contains a center position offset. Width and height Confidence level and category prediction By traversing the image grid cells and normalizing the predicted offsets to map them back to the original image space, the center coordinates and corresponding attributes of the candidate target boxes are obtained, achieving preliminary perception of the target. The confidence filtering branch performs a decision operation based on the perceived information, selecting the predicted box with the highest confidence as the final candidate box; if the confidence of all candidate target boxes is lower than a preset threshold... If the previous method failed, it degenerates into using the entire region corresponding to the localization feature map as the final candidate box, thus ensuring the robustness of the candidate box generation process. The cropping box generation branch performs spatial cropping based on the center point of the final candidate box output by the confidence filtering branch, using the center of the final candidate box as the center point. Construct a square cutting area with fixed side lengths based on the baseline; side length The size of the input image to be segmented With cutting proportions The product is determined, that is At the same time, the predicted width and height... Constraints are imposed to ensure the cropping results remain within a reasonable range, avoiding distortion caused by abnormal predictions. The boundary compensation branch calculates the out-of-bounds portion and performs translation compensation when the cropping box coordinates exceed the image boundary, ensuring the cropping box re-falls into the effective image area while maintaining its original size, thus guaranteeing the consistency and effectiveness of spatial guidance.

[0098] In some embodiments, steps S140-S150 may specifically include:

[0099] The target attention bounding box generated in the detection phase is combined with the image to be segmented input for the segmentation task. The region where the disguised object is located is cropped and magnified at multiple magnifications to obtain a set of multi-scale perceptual attention region image patches. The category code generated in the classification phase is input as an additional semantic channel into the backbone network, and the backbone network extracts semantic features from the multi-scale perceptual attention region image patches to obtain a first semantic feature map, a second semantic feature map, a third semantic feature map, and a fourth semantic feature map. The multi-level semantic feature maps are input into a multi-guided feature fusion module to achieve the fusion of multi-source information and feature reconstruction, thereby obtaining a fine-grained feature representation of the disguised object. Finally, the multi-source fine-grained feature representation is input into a multi-task perceptual decoder, and a refined segmentation prediction result is output through a grouped feature interaction and progressive fusion mechanism, while maintaining the consistency of the semantic structure.

[0100] The first semantic feature map, the second semantic feature map, the third semantic feature map, and the fourth semantic feature map correspond to the semantic feature results output at different levels in the Transformer backbone network, respectively, and are used to characterize the feature information of the multi-scale perceptual attention region at different semantic levels.

[0101] The backbone network is used to extract semantic features from the cropped image patches and category codes input in the segmentation stage; the multi-guided feature fusion module is used to fuse and reconstruct multi-source semantic features to obtain preliminary segmentation results; the multi-task perceptual decoder is used to optimize and decode based on the preliminary segmentation results using a multi-task reconstruction and feature interaction mechanism to obtain the final camouflaged object segmentation results.

[0102] The backbone network is composed of a Transformer backbone network, which includes, but is not limited to, any of the following implementations: a PVTv2 network based on a multi-head self-attention mechanism, a feedforward neural network, layer normalization, and residual connections, or its equivalent or improved structure.

[0103] The multi-guided feature fusion module includes a feature alignment and interaction submodule and a feature reconstruction and enhancement submodule. The feature alignment and interaction submodule includes a feature preprocessing branch, a spatial alignment branch, and a multi-head attention branch. The feature preprocessing branch performs normalization operations on the input multi-scale and multi-source features, and applies convolution operations with kernels of a preset size to achieve consistent feature distribution and suppress redundant information. The spatial alignment branch generates spatial description information at different scales through three parallel max-pooling layers and a global average pooling layer, followed by sequential convolutional layers, normalization layers, and ReLU activation functions to achieve adaptive adjustment of spatial resolution and scale feature alignment. The multi-head attention branch utilizes a multi-head self-attention mechanism to capture long-range dependencies and cross-scale feature interaction information, and enhances the stability of feature modeling with normalization layers and skip connections.

[0104] The feature reconstruction and enhancement submodule includes a multi-receptive-field convolutional branch and a gated activation branch. The multi-receptive-field convolutional branch uses three deep convolutional layers with different kernels to jointly represent global and local semantic information. The gated activation branch dynamically adjusts the importance of feature channels and spatial locations through convolution, normalization, GeLU activation, and concatenation operations, achieving information enhancement and noise suppression during the fusion process.

[0105] The multi-task perceptual decoder consists of a group interaction submodule and a progressive fusion submodule. The group interaction submodule includes a feature unrolling unit, a feature grouping unit, and an intra-group interaction unit. The feature unrolling unit processes the input features sequentially through multi-dilated convolutional layers and divides the processing results into several subsets according to channels. The feature grouping unit processes each subset group by concatenating the input of the current subset with the output of the previous group, followed by convolution, normalization, and ReLU activation to obtain the corresponding intra-group output features. The intra-group interaction unit further models the internal relationships of the intra-group output features to enhance their expressive power.

[0106] The progressive fusion submodule includes a feature reconstruction unit, a fusion generation unit, and a residual output unit. Specifically, the feature reconstruction unit divides the explicit features obtained from the group interaction unit into two subsets, which are then input into a multi-head self-attention mechanism to model cross-region dependencies and obtain two sets of enhanced features. The fusion generation unit includes sequentially connected convolutional kernels... The convolutional layers, normalized layers, and ReLU activation function are used. The residual output unit includes skip connections to maintain continuity between the original input information and the fused features; and sequentially connected convolutional kernels... The convolutional layers, normalization layers, and ReLU activation function.

[0107] The parameters of the backbone network are initialized using the weights of a PVTv2 or ResNet model obtained through fully supervised pre-training on the ImageNet dataset, thereby providing more discriminative feature representations for subsequent tasks. For training optimization, this embodiment uses the AdamW optimizer to train the model, with a total of 150 training epochs, an initial learning rate of 0.0001, a batch size of 4, and a weight decay coefficient of 0.0001 to ensure a balance between convergence speed and generalization ability.

[0108] In the implementation of this invention, to segment camouflaged objects in an image, the input image is first provided to the model, which then outputs the pixel regions corresponding to the camouflaged objects. Specifically, in the classification stage, a coarse feature extractor with ResNet or Transformer as the backbone network is chosen because the semantic features encoded by such backbone networks are beneficial for the model to fully acquire the category information and salient features of the camouflaged objects, thereby improving the recognition effect of the camouflaged regions. Furthermore, the K-Means classifier is introduced under unsupervised conditions because this classifier can effectively distinguish the potential categories of camouflaged objects by minimizing the intra-class sum of squares error.

[0109] In the detection phase, since object detection requires precise localization of camouflaged objects, a Transformer-based object detector is employed to leverage its semantic feature encoding advantages, thereby enhancing the modeling ability of camouflaged object location information. Furthermore, an adaptive candidate strategy is used because it dynamically selects the optimal target attention box within the camouflaged object region, generating more accurate candidate regions during detection. In the segmentation phase, the Transformer is continued as the backbone network because its ability to express contextual semantic features helps the model capture the complex relationships between the camouflaged object and its surrounding environment, thus improving segmentation accuracy.

[0110] In the segmentation stage, a multi-guided feature fusion module is included because fusing multi-scale and multi-guided features through feature alignment and interaction mechanisms enables full complementarity of information. Furthermore, the designed multi-head attention branch captures long-range dependencies and cross-scale feature interaction information, thereby enhancing model stability and preventing gradient vanishing. Simultaneously, the feature reconstruction and enhancement module reconstructs and enhances multi-task features through multi-receptive field convolution and dynamic gating activation, facilitating full fusion of multi-task information and achieving more accurate feature guidance. Finally, in the decoding stage, a multi-task perceptual decoder is employed because its feature grouping unit and intra-group interaction unit can fully explore the structural spatial features and fine-grained local features of multi-task information; combined with a multi-head self-attention mechanism based on feature reconstruction, it improves cross-region feature modeling capabilities and strengthens grouped information representation, thus making the model more accurate in distinguishing between camouflaged objects and boundary-confusing regions.

[0111] The solution of this disclosure achieves accurate segmentation of camouflaged objects by combining the correlation between classification, detection and segmentation tasks in the camouflage object segmentation task. It can effectively distinguish camouflage objects from background areas, and shows high accuracy and generalization, especially in complex camouflage environments.

[0112] As shown in Figure 2, the technical solution of this embodiment of the invention divides the semantic segmentation task of camouflaged objects into three processing stages: image classification, image detection, and image segmentation. The image classification stage is as follows... Figure 2A As shown, corresponding to the aforementioned step S120; the image detection stage is as follows: Figure 2B As shown, corresponding to the aforementioned step S130; the image segmentation stage is as follows Figure 2C As shown in (c), this corresponds to the aforementioned steps S140-S150. (d), (e), and (f) are detailed explanations of some functions in the segmentation stage. Figure 2C In step (d), the category code obtained in stage (a) is added to the segmentation stage. Specifically, the backbone network of the segmentation stage is expanded from three channels to four channels, with the fourth channel being the category information. Figure 2D (e) in the text represents the multi-guided feature fusion module, such as... Figure 2E In this context, (f) represents the multi-task-aware decoder. For example... Figure 2F This represents a legend.

[0113] Table 1 presents a quantitative comparison between the technical solution proposed in this disclosure and sixteen existing advanced semantic segmentation methods for camouflaged objects. It can be seen that this disclosure achieves superior performance across multiple metrics.

[0114] Furthermore, Figure 3This is the image to be segmented input in one embodiment of this disclosure; Figure 4 This is a schematic diagram of a pixel-level label in one embodiment of the present disclosure; Figures 5 to 8 The paper presents a qualitative comparison between the technical solution of this disclosure and three current advanced camouflage object semantic segmentation methods (CamoDiffusion, VSCode, and ZoomNeXt). The comparison shows that this disclosure achieves more accurate and robust camouflage object segmentation in complex backgrounds.

[0115] Table 1. Quantitative comparison results between this embodiment and 16 state-of-the-art semantic segmentation models for camouflaged objects. This represents a structural metric used to measure the consistency between the predicted mask and the true mask in terms of overall structure and regional distribution. Indicates weighted The value is used to measure the weighted balance between the model's precision and recall at the pixel level. This represents an enhanced consistency metric, used to measure the similarity between the predicted map and the ground truth map in terms of both global statistical distribution and local pixel matching. This represents the mean absolute error, used to calculate the average pixel difference between the predicted image and the true mask.

[0116] Table 1

[0117]

[0118] Based on the same inventive concept, this disclosure provides a semantic segmentation device for camouflaged objects based on an adaptive candidate strategy. The steps performed by the components of this device are the same as or similar to those described above, therefore, similar details will not be repeated. Figure 9 As shown, a semantic segmentation device for camouflaged objects based on an adaptive candidate strategy in this embodiment includes:

[0119] Image acquisition module 910 is used to acquire the image to be segmented.

[0120] The initial classification module 920 is used to extract multi-scale features from the image to be segmented using a coarse feature extractor to obtain multiple classification feature maps at different scales, and to cluster the multiple classification feature maps at different scales using a classifier to obtain the category code of the image to be segmented.

[0121] The target determination module 930 is used to encode and extract localization features of the image to be segmented using a target detector to obtain multiple localization feature maps of different scales, and to dynamically analyze the multiple localization feature maps of different scales using an adaptive candidate strategy to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects.

[0122] The image enhancement module 940 is used to crop the image to be segmented based on the target attention box to obtain cropped image blocks, and to magnify the cropped image blocks by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks.

[0123] The deep classification module 950 is used to input the category encoding and the multi-scale perceptual attention region image patch into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. The multi-guided feature fusion module is used to perform feature fusion and feature reconstruction on the multiple semantic feature maps at different scales to obtain multiple fine-grained feature representations of the disguised object at different scales. The multi-task perceptual decoder is used to perform group feature interaction and stepwise fusion on the multiple fine-grained feature representations at different scales, and the segmentation result of the disguised object is determined based on the fusion result.

[0124] According to embodiments of this disclosure, this disclosure also provides a computer-readable storage medium, wherein the computer program, when executed by a processor, implements the camouflaged object semantic segmentation method based on an adaptive candidate strategy of any of the above embodiments, and repeated parts will not be described again.

[0125] The various embodiments of the techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0129] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0130] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0131] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A semantic segmentation method for camouflaged objects based on an adaptive candidate strategy, characterized in that, include: Obtain the image to be segmented; The image to be segmented is subjected to multi-scale feature extraction using a coarse feature extractor to obtain multiple classification feature maps at different scales. Then, the multiple classification feature maps at different scales are clustered using a classifier to obtain the category code of the image to be segmented. The image to be segmented is encoded and localization features are extracted using an object detector to obtain multiple localization feature maps of different scales. An adaptive candidate strategy is then used to dynamically parse the multiple localization feature maps of different scales to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects. The image to be segmented is cropped based on the target attention box to obtain cropped image blocks, and the cropped image blocks are magnified by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks. The category encoding and the multi-scale perceptual attention region image patch are input into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. The multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales to obtain fine-grained feature representations of the disguised object at multiple different scales. A multi-task perceptual decoder is used to group and interact features of the multiple fine-grained feature representations at different scales and then gradually fuse them. Based on the fusion result, the segmentation result of the camouflaged object is determined. The multi-guided feature fusion module includes a feature alignment and interaction submodule and a feature reconstruction and enhancement submodule; The feature alignment and interaction submodule includes a feature preprocessing branch, a spatial alignment branch, and a multi-head attention branch; The feature preprocessing branch is used to normalize semantic feature maps of multiple different scales and apply convolution operations with kernels of a preset size to achieve consistency of feature distribution and suppression of redundant information. The spatial alignment branch is used to process the features output by the feature preprocessing branch through three parallel max pooling layers and a global average pooling layer to generate spatial description information at different scales; and to process the spatial description information at different scales by sequentially passing through convolutional layers, normalization layers and ReLU activation functions to achieve adaptive adjustment of spatial resolution and scale feature alignment. The multi-head attention branch is used to process the features output by the spatial alignment branch using a multi-head self-attention mechanism to capture long-range dependencies and cross-scale feature interaction information, and to obtain a stable feature representation with enhanced features by combining a normalization layer and skip connections. The feature reconstruction and enhancement submodule includes a multi-receptive-field convolutional branch and a gated activation branch; The multi-receptive-field convolutional branch is used to process the stable feature representation through a deep convolutional layer with three different convolutional kernels to obtain a joint feature representation of global and local semantic information. The gated activation branch is used to process the joint feature representation of global and local semantic information through convolution, normalization layer, GeLU activation and concatenation operations, so as to realize dynamic control of the importance of feature channels and spatial location, as well as information enhancement and noise suppression in the fusion process, and obtain fine-grained feature representations of camouflaged objects at multiple different scales.

2. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The coarse feature extractor includes a ResNet backbone network or a Transformer backbone network; The ResNet backbone network is based on a residual block stacking structure, and the residual block includes two convolutional layers, a batch normalization layer, and a ReLU activation function; The Transformer backbone network includes a PVTv2 network, which includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

3. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The classifier includes a module for minimizing the intra-class squared distance error and a module for iteratively updating cluster centers; The module for minimizing intra-class squared distance error is used to calculate the distance between the input feature vector and the current cluster center based on any of the metrics such as Euclidean distance, cosine similarity, or Mahalanobis distance, and to assign the feature vector to the cluster to which the cluster center with the smallest distance belongs, thereby minimizing the intra-class sum of squares error. The iterative cluster center update module is used to recalculate the mean of all feature vectors in each cluster after completing a feature vector classification, and use the new mean as the new cluster center.

4. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The target detector includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network, and the PVTv2 network includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

5. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The module that executes the adaptive candidate strategy includes a prediction tensor parsing branch, a confidence filtering branch, a clipping box generation branch, and a boundary compensation branch. The prediction tensor parsing branch is used to perceive the spatial and semantic information of potential camouflaged objects from the input localization feature map. The prediction tensor is obtained through an end-to-end detection model. The prediction tensor includes the position information of the corresponding potential camouflaged object and the predicted category. The position information includes the center position offset, width and height information, and confidence score of the potential camouflaged object. By traversing the grid cells of the localization feature map, the center position offset in the prediction tensor is normalized and mapped to the space of the image to be segmented, and the center coordinates of the candidate target box are obtained by combining the width and height information. The confidence filtering branch is used to select the candidate target boxes with the highest confidence and a confidence level higher than a preset threshold as the final candidate boxes; If the confidence scores of all candidate bounding boxes are lower than the preset threshold, then the bounding boxes corresponding to the entire localization feature map are used as the final candidate bounding boxes. The clipping box generation branch is used to construct a clipping box that meets the side length constraints based on the center coordinates of the final candidate box; The boundary compensation branch is used to calculate the position information of the out-of-bounds portion when the coordinates of the cropping box exceed the boundary of the image to be segmented, and to perform translation compensation based on the position information so that the cropping box falls back into the effective area of ​​the image to be segmented while maintaining its original size, thus obtaining the target attention box.

6. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The backbone network includes a Transformer backbone network; the Transformer backbone network includes a PVTv2 network; the PVTv2 network includes a multi-head self-attention mechanism, a feedforward neural network, a normalization layer, and residual blocks.

7. The semantic segmentation method for camouflaged objects based on an adaptive candidate strategy according to claim 1, characterized in that, The multi-task perception decoder includes a group interaction submodule and a progressive fusion submodule; The grouping interaction submodule includes a feature expansion unit, a feature grouping unit, and an intra-group interaction unit; The feature unrolling unit is used to process multiple fine-grained feature representations of different scales of input sequentially through a multi-hole convolutional layer, and divide the processing results into several subsets according to channels; The feature grouping unit is used to process each subset one by one. The input of the current subset is concatenated with the output of the previous group and then passed through convolution, normalization layer and ReLU activation in sequence to obtain the corresponding in-group output features. The intra-group interaction unit is used to perform internal correlation modeling on the intra-group output features to obtain explicit features; The progressive fusion submodule includes: a feature reconstruction unit, a fusion generation unit, and a residual output unit; The feature reconstruction unit is used to divide the explicit features obtained by the intra-group interaction unit into two subsets, which are then input into the multi-head self-attention mechanism to model cross-regional dependencies and obtain two sets of enhanced features. The fusion generation unit utilizes sequentially connected convolutional kernels as... The convolutional layers, normalization layers, and ReLU activation function are used to process the two sets of enhanced features to obtain fused features; The residual output unit is used for skip connections and convolution kernels. The convolutional layers, normalization layers, and ReLU activation functions process the fused features to obtain the final fused features; the skip connections are used to maintain the continuity between the original input information of the multi-task perceptual decoder and the final fused features.

8. A semantic segmentation device for camouflaged objects based on an adaptive candidate strategy, characterized in that, include: The image acquisition module is used to acquire the image to be segmented; The initial classification module is used to extract multi-scale features from the image to be segmented using a coarse feature extractor to obtain multiple classification feature maps at different scales, and to cluster the multiple classification feature maps at different scales using a classifier to obtain the category code of the image to be segmented. The target determination module is used to encode and extract localization features of the image to be segmented using a target detector to obtain multiple localization feature maps of different scales, and to dynamically parse the multiple localization feature maps of different scales using an adaptive candidate strategy to determine the candidate target boxes and target attention boxes corresponding to the camouflaged objects. The image enhancement module is used to crop the image to be segmented based on the target attention box to obtain cropped image blocks, and to magnify the cropped image blocks by multiple magnifications to obtain a set of multi-scale perceptual attention region image blocks. A deep classification module is used to input the category code and the multi-scale perceptual attention region image patch into the backbone network. The backbone network extracts multi-scale semantic features from the input information to obtain multiple semantic feature maps at different scales. A multi-guided feature fusion module is used to fuse and reconstruct the multiple semantic feature maps at different scales to obtain fine-grained feature representations of the disguised object at multiple different scales. The multi-task perceptual decoder is used to group and interact the fine-grained feature representations of multiple different scales and gradually fuse them, and the segmentation result of the disguised object is determined based on the fusion result. The multi-guided feature fusion module includes a feature alignment and interaction submodule and a feature reconstruction and enhancement submodule; The feature alignment and interaction submodule includes a feature preprocessing branch, a spatial alignment branch, and a multi-head attention branch; The feature preprocessing branch is used to normalize semantic feature maps of multiple different scales and apply convolution operations with kernels of a preset size to achieve consistency of feature distribution and suppression of redundant information. The spatial alignment branch is used to process the features output by the feature preprocessing branch through three parallel max pooling layers and a global average pooling layer to generate spatial description information at different scales; and to process the spatial description information at different scales by sequentially passing through convolutional layers, normalization layers and ReLU activation functions to achieve adaptive adjustment of spatial resolution and scale feature alignment. The multi-head attention branch is used to process the features output by the spatial alignment branch using a multi-head self-attention mechanism to capture long-range dependencies and cross-scale feature interaction information, and to obtain a stable feature representation with enhanced features by combining a normalization layer and skip connections. The feature reconstruction and enhancement submodule includes a multi-receptive-field convolutional branch and a gated activation branch; The multi-receptive-field convolutional branch is used to process the stable feature representation through a deep convolutional layer with three different convolutional kernels to obtain a joint feature representation of global and local semantic information. The gated activation branch is used to process the joint feature representation of global and local semantic information through convolution, normalization layer, GeLU activation and concatenation operations, so as to realize dynamic control of the importance of feature channels and spatial location, as well as information enhancement and noise suppression in the fusion process, and obtain fine-grained feature representations of camouflaged objects at multiple different scales.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images