A deep intelligent segmentation method based on inductive bias and dynamic feature aggregation

Abstract

The present invention relates to the field of computer vision technology, and in particular to a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, including: constructing an image intelligent segmentation model, including an initial layer, an encoder, a bottleneck layer, a decoder, a spatial attention module and an output layer; inputting the original image into the image intelligent segmentation model, and outputting a segmentation prediction image. The image intelligent segmentation model uses a DConvNeXt module, combining a deformable convolution v4 operator with dynamic feature modeling capabilities and an efficient local attention mechanism as a token mixer, which effectively improves the feature modeling capabilities of the model while significantly enhancing the flexibility and dynamics of feature representation. The present invention can effectively handle complex image segmentation tasks, significantly improves the adaptability in diversified application scenarios and the segmentation accuracy of processing complex images.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation. Background Art

[0002] In recent years, the rapid development of artificial intelligence technology has greatly promoted the technological progress and application innovation of intelligent systems. Deep image intelligent segmentation technology simulates the analysis process of human vision, uses advanced deep learning algorithms to analyze and process image data, identifies and separates different objects and areas from complex visual scenes, and realizes automatic image segmentation, showing broad application prospects in multiple industries. In the field of smart medical care, this technology can automatically analyze medical images, accurately segment lesion areas, and assist doctors in diagnosis and treatment; in the field of industrial inspection, this technology can automatically detect product defects and improve the automation level and product quality of production lines; in the field of autonomous driving, this technology analyzes road images in real time, segmenting roads, vehicles and pedestrians, etc., to improve the perception ability and safety of autonomous driving vehicles.

[0003] At present, deep image intelligent segmentation technology has significantly improved the accuracy and efficiency of image segmentation tasks. Its core algorithms mainly include methods based on convolutional neural networks (CNNs) and methods based on Transformers. CNN methods are represented by classic encoder-decoder models such as FCN, SegNet, and U-Net. Subsequently, many works have further improved the segmentation level by introducing improved convolution modules, residual structures, attention mechanisms, and multi-scale feature aggregation methods. However, these methods are limited by the local receptive field of convolution and are difficult to effectively learn long-distance dependencies that are crucial to generating accurate segmentation. The Transformer architecture uses the self-attention mechanism to achieve dynamic feature aggregation and global receptive field, and can model powerful feature representations, but lacks the inherent inductive bias of convolutional networks and relies on pre-training on large-scale data. Therefore, many methods choose to use a CNN-Transformer hybrid architecture to fuse local and global features to improve model performance, such as SETR, SegFormer, and TransUNet. However, these methods are difficult to play their advantages in application scenarios where high-quality labeled data is scarce, and they also face high computational overhead.

[0004] It can be seen that the existing deep image intelligent segmentation models are limited by the inherent characteristics of their architectures, making it difficult to adapt to a variety of application scenarios and unable to fully meet the image segmentation needs in different fields. In addition, existing methods have difficulty maintaining good segmentation performance when processing complex images, especially when facing targets with complex shapes and irregular forms, and cannot meet the needs of accurate segmentation in practical applications. These methods are often limited by local receptive fields or rely on large-scale data sets, and it is difficult to achieve efficient and accurate segmentation in complex scenes, data-scarce or computing resource-constrained environments, which directly affects the performance of image segmentation intelligent systems and limits their widespread deployment in different fields. Summary of the invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of low segmentation accuracy of complex images in the prior art.

[0006] To solve the above technical problems, the present invention provides a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, comprising:

[0007] Build an intelligent image segmentation model, including the initial layer, encoder, bottleneck layer, decoder, spatial attention module, and output layer;

[0008] Input the original image into the image intelligent segmentation model and output the segmentation prediction image, including:

[0009] Input the original image into the initial layer of the image intelligent segmentation model to obtain the initial feature map;

[0010] The initial feature map is input to the encoder to obtain the encoding feature map of each stage; the encoder includes four stages, each stage includes a DConvNeXt module and a downsampling module connected along the forward propagation direction; the output of the DConvNeXt module of each stage in the encoder is the encoding feature map of the stage;

[0011] The DConvNeXt module includes a token mixer, a normalization layer, and a multilayer perceptron connected in sequence; the output features of the multilayer perceptron are added to the input features of the DConvNeXt module to obtain the output features of the DConvNeXt module; the token mixer includes a deformable convolution v4 operator and an efficient local attention mechanism connected in sequence, and the output features of the deformable convolution v4 operator are added to the output features of the efficient local attention mechanism to obtain the output features of the token mixer;

[0012] Input the encoded feature map of the last stage in the encoder into the bottleneck layer to obtain the bottleneck layer output feature map;

[0013] The output feature map of the bottleneck layer is input to the decoder to obtain a decoded feature map; the decoder includes four stages, each of which includes an upsampling module and a DConvNeXt module connected along the forward propagation direction; in the decoder, the output feature map of the upsampling module of each stage and the encoding feature map of the corresponding stage of the encoder are input to the spatial attention module through a jump connection, and the output feature map of the spatial attention module is added to the output feature map of the upsampling module of this stage as the input of the DConvNeXt module of this stage; the output of the DConvNeXt module of the last stage in the decoder is a decoding feature map;

[0014] The decoded feature map is input into the output layer to obtain the segmentation prediction image.

[0015] Preferably, the process of the DConvNeXt module is expressed as:

[0016] ;

[0017] ;

[0018] ;

[0019] in, represents the input features of the DConvNeXt module, represents the output features of the DConvNeXt module, represents the deformable convolution v4 operator, represents the output features of the deformable convolution v4 operator, represents an efficient local attention mechanism, represents the output characteristics of the token mixer, represents the normalization layer, represents a 1×1 convolution used to expand the number of channels from C to R times. represents the GELU activation function, Represents a 1×1 convolution used to restore the number of channels from CR to C.

[0020] Preferably, the deformable convolution v4 operator is calculated on the input feature map through a sliding window, and the calculation process is expressed as:

[0021] ;

[0022] in, represents the sub-region of the input feature map of the deformable convolution v4 operator, represents the convolution operation output of the sub-region, Represents the sampling center point of the sub-region of the input feature map; Indicates the number of groups of channels. Indicates The sub-feature graph corresponding to each group, Number of channels , Indicates the number of channels of the input feature map; Indicates The projection weights shared by the groups; Indicates the number of sampling points; Relative to the sampling center point No. The location coordinates of the sampling points; Indicates Sampling points in group The corresponding offset, Indicates In the group The modulation scalar of the sampling points.

[0023] Preferably, the calculation process of the efficient local attention mechanism is expressed as:

[0024] ;

[0025] ;

[0026] ;

[0027] in, represents the input feature map of the efficient local attention mechanism, represents the output feature map of the efficient local attention mechanism, and Represent the attention weights in the horizontal and vertical directions respectively, represents the Sigmoid activation function, represents group normalization, represents one-dimensional convolution, and Represents the average pooling operation in the horizontal and vertical directions respectively.

[0028] Preferably, the processing process of the spatial attention module includes:

[0029] The output feature map of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are input into the spatial attention module. The output feature map of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are respectively subjected to a 1×1 convolution layer, adjusted to the same number of feature channels C, and then added to obtain a fused feature map; the fused feature map is sequentially subjected to the ReLU activation function, 1×1 convolution layer and normalization layer, and Sigmoid activation function to generate an attention weight map representing the importance of each spatial position. The attention weight map is multiplied by the input encoded feature map to obtain the output feature map of the spatial attention module.

[0030] Preferably, the image intelligent segmentation model applies a deep supervision strategy to optimize the training process, and uses the output of the bottleneck layer and the output of each stage of the decoder as output branches to independently calculate the loss; the formula of the loss function is:

[0031] ;

[0032] in, is the total loss of the image intelligent segmentation model, and n is the number of deep supervision output branches; is the weight coefficient of the i-th output branch, is the loss value of the i-th output branch.

[0033] Preferably, the input features of the downsampling module are sequentially passed through a deep convolution layer, a normalization layer, a first convolution layer, a GELU activation function, and a second convolution layer to obtain a first downsampling feature. The input features of the downsampling module are passed through a third convolution layer to obtain a second downsampling feature. The output features of the downsampling module are obtained by adding the first downsampling feature and the second downsampling feature.

[0034] Preferably, the input features of the upsampling module are sequentially passed through a first transposed convolutional layer, a normalization layer, a fourth convolutional layer, a GELU activation function, and a fifth convolutional layer to obtain a first upsampling feature, and the input features of the upsampling module are passed through a second transposed convolutional layer to obtain a second upsampling feature; the first upsampling feature and the second upsampling feature are added to obtain the output feature of the upsampling module.

[0035] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0036] The present invention discloses a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, constructs an image intelligent segmentation model and applies a DConvNeXt module in the image intelligent segmentation model. The DConvNeXt module combines a deformable convolution v4 operator with dynamic feature modeling capability and an efficient local attention mechanism as a token mixer, which effectively improves the feature modeling capability of the model and significantly enhances the flexibility and dynamics of feature representation. The deformable convolution v4 operator dynamically adjusts the sampling position and sampling weight according to the input, and can flexibly cope with target areas with complex shapes. The combination of the efficient local attention mechanism enables the model to further focus on key information, and can effectively capture the shape perception features of the target area.

[0037] In addition, the image intelligent segmentation model introduces a spatial attention module on the skip connection path, which can effectively optimize the features transmitted in the encoder stage and enhance key spatial position information.

[0038] Therefore, the present invention can effectively handle complex image segmentation tasks, significantly improves the adaptability in diversified application scenarios and the segmentation accuracy of complex images, and provides a novel and efficient solution for image segmentation services in different fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0040] Figure 1 It is the structural diagram of the image intelligent segmentation model;

[0041] Figure 2 It is the structure diagram of Transformer module, ConvNeXt module and DConvNeXt module, where Figure 2 (a) is the Transformer module and its macro structure diagram. Figure 2 (b) is the ConvNeXt module and its macro structure diagram. Figure 2 (c) is the structural diagram of the DConvNeXt module;

[0042] Figure 3 This is a diagram of the structure of an efficient local attention mechanism;

[0043] Figure 4 It is the structural diagram of the spatial attention module;

[0044] Figure 5 This is the structural diagram of the ConvNeXt downsampling module;

[0045] Figure 6This is the structural diagram of the ConvNeXt upsampling module;

[0046] Figure 7 It is the ablation experiment result diagram of different module components. DETAILED DESCRIPTION

[0047] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0048] Embodiment 1

[0049] The present invention provides a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, comprising:

[0050] Construct an image intelligent segmentation model, the structure of which refers to Figure 1 As shown, it includes the initial layer, encoder, bottleneck layer, decoder, spatial attention module and output layer;

[0051] Input the original image into the image intelligent segmentation model and output the segmentation prediction image, including:

[0052] Input the original image into the initial layer of the image intelligent segmentation model to obtain the initial feature map;

[0053] The initial feature map is input to the encoder to obtain the encoding feature map of each stage; the encoder includes four stages, each stage includes a DConvNeXt module and a downsampling module connected along the forward propagation direction; the output of the DConvNeXt module of each stage in the encoder is the encoding feature map of the stage;

[0054] Input the encoded feature map of the last stage in the encoder into the bottleneck layer to obtain the bottleneck layer output feature map;

[0055] The output feature map of the bottleneck layer is input to the decoder to obtain a decoded feature map; the decoder includes four stages, each of which includes an upsampling module and a DConvNeXt module connected along the forward propagation direction; in the decoder, the output feature map of the upsampling module of each stage and the encoding feature map of the corresponding stage of the encoder are input to the spatial attention module through a jump connection, and the output feature map of the spatial attention module is added to the output feature map of the upsampling module of this stage as the input of the DConvNeXt module of this stage; the output of the DConvNeXt module of the last stage in the decoder is a decoding feature map;

[0056] The decoded feature map is input into the output layer to obtain the segmentation prediction image.

[0057] The initial layer is a 1×1 convolutional layer.

[0058] The output layer is a 1×1 convolutional layer.

[0059] Specifically, the DConvNeXt module is an advanced design that integrates the ConvNeXt module and the Transformer module. The structures of the Transformer module, the ConvNeXt module, and the DConvNeXt module refer to Figure 2 As shown, Figure 2 (a) is the Transformer module and its macro structure diagram. Figure 2 (b) is the ConvNeXt module and its macro structure diagram. Figure 2 (c) in the figure is the structural diagram of the DConvNeXt module. The DConvNeXt module includes a token mixer, a normalization layer, and a multilayer perceptron connected in sequence. The output features of the multilayer perceptron are added to the input features of the DConvNeXt module to obtain the output features of the DConvNeXt module.

[0060] The ConvNeXt module is an advanced structure that combines the advantages of CNN and Transformer architectures. It simplifies the components while retaining the core architecture of Transformer, and uses 7×7 large-kernel deep convolution as a token mixer to model long-distance dependencies, performing well in various visual tasks.

[0061] This paper follows the design concept of the ConvNeXt module and redesigns a more powerful token mixer based on its macro structure, that is, combining the Deformable Convolution v4 (DCNv4) operator with the Efficient Local Attention (ELA) mechanism to build a key token mixer.

[0062] The token mixer of the DConvNeXt module includes a deformable convolution v4 operator and an efficient local attention mechanism connected in sequence, wherein the output features of the deformable convolution v4 operator are added to the output features of the efficient local attention mechanism to obtain the output features of the token mixer.

[0063] Input features of DConvNeXt module After feature modeling by DCNv4, we get , and then the output characteristics of the token mixer are obtained through ELA Next, the output characteristics of the token mixer are After layer normalization, it is input into a multi-layer perceptron with an inverse bottleneck design to further model rich feature representation. This embodiment uses 1×1 convolution to build a multi-layer perceptron. The input features of the multi-layer perceptron are expanded to R times the original number of channels C through the first convolution layer, that is, CR. Then, the channels are reduced through the second convolution layer, and the number of channels is restored to C. Finally, the features with restored channel numbers are compared with the input features of the DConvNeXt module. Perform residual connection to get the output Y. This process is expressed as:

[0064] ;

[0065] ;

[0066] ;

[0067] in, represents the input features of the DConvNeXt module, represents the output features of the DConvNeXt module, represents the deformable convolution v4 operator, represents the output features of the deformable convolution v4 operator, represents an efficient local attention mechanism, represents the output characteristics of the token mixer, represents the normalization layer, represents a 1×1 convolution used to expand the number of channels from C to R times. represents the GELU activation function, Represents a 1×1 convolution used to restore the number of channels from CR to C.

[0068] The DConvNeXt module effectively improves the model's adaptability to deformed objects in images by introducing deformable convolutions with dynamic feature aggregation capabilities, and further enhances the model's ability to focus on key information by combining efficient local attention mechanisms. The DConvNeXt module not only inherits the inductive bias and easy-to-train characteristics of the convolutional structure, but also has the dynamic feature aggregation capabilities of the Transformer, significantly improving the model's flexibility in handling different image segmentation tasks, as well as the segmentation accuracy of complex images and irregular targets.

[0069] Specifically, DCNv4 is the core component in the DConvNeXt module, and this operator is an effective mechanism for adapting to deformed objects. The offset and modulation scalar of the sampling point are dynamically adjusted according to the input to achieve flexible sampling positions and unlimited weight ranges. This enables the model to achieve dynamic feature modeling, adaptively aggregate information at different spatial positions, flexibly learn the appropriate receptive field range, and effectively perceive the shape characteristics of irregular target areas. Referring to the design of multi-head self-attention, DCNv4 groups modeling feature representations. Each group uses independent offsets and modulation scalars, which can learn rich information from different feature representation subspaces, which is conducive to extracting key context from complex images. Different sampling points in each group use shared projection weights to effectively reduce computational complexity. In addition, the design of DCNv4 simplifies the calculation process, avoids unnecessary computational burden, and can maintain efficient operation. DCNv4 calculates on the input feature map through a sliding window, and the calculation process is expressed as:

[0070] ;

[0071] in, represents the sub-region of the DCNv4 input feature map, represents the convolution operation output of the sub-region, Represents the sampling center point of the sub-region of the input feature map; Indicates the number of groups of channels. Indicates The sub-feature graph corresponding to each group, Number of channels , Indicates the number of channels of the input feature map; Indicates The projection weights shared by the groups; Indicates the number of sampling points, which is 9 in this embodiment; The sampling points are It spreads regularly around the center. Relative to the sampling center point No. The position coordinates of the sampling points; in this embodiment, The coordinates of are (0,0), then ; Indicates Sampling points in group The corresponding offset,after adding the offset to each sampling point, an irregularly shaped,sampling grid is formed; Indicates In the group The modulation scalar of a sampling point, that is, the weight of the sampling point, is used to distinguish the importance of different spatial positions.

[0072] After DCNv4, an efficient local attention mechanism is applied to further optimize the output features of DCNv4, so that the model can focus more on the target area without significantly increasing the computational cost. ELA integrates spatial position information into modeling channel dimension information to achieve attention weighting of input features in both channel and spatial dimensions. The structure of ELA is as follows Figure 3 As shown in the figure, global average pooling operations are applied to the input feature map in the horizontal and vertical directions respectively to obtain sub-feature maps in two dimensions. The sub-feature map is then modeled by a shared one-dimensional convolution. Here, a one-dimensional convolution with a large kernel, that is, a convolution with a kernel size of 7, is applied to enhance the feature modeling capability. After that, a channel attention map in two spatial dimensions is generated through a normalization layer and an activation function. Here, group normalization is chosen to avoid the performance degradation that may occur in batch normalization in small batches of data. Finally, the generated channel attention map is multiplied with the input feature map to obtain a weighted feature representation. The formula is expressed as:

[0073] ;

[0074] ;

[0075] ;

[0076] in, represents the input feature map of ELA, represents the output feature map of ELA, and Represent the attention weights in the horizontal and vertical directions respectively, represents the Sigmoid activation function, Represents group normalization (GroupNormalization), represents one-dimensional convolution, and Represents the average pooling operation in the horizontal and vertical directions respectively.

[0077] Compared with the classic channel attention method SE module, ELA's structure does not have a channel dimension reduction process, avoiding information loss. Compared with the widely used hybrid attention method CBAM, the structure is simpler and lighter, and performs better in terms of operating efficiency. In terms of overall performance, ELA shows superior results, can effectively enhance the key information in the feature map, and further improve the feature modeling capabilities of the DConvNeXt module.

[0078] In the image intelligent segmentation model, a spatial attention gate (AG) is added to each jump connection path of the encoder and decoder to optimize the feature map passed by the encoder stage and introduce more relevant low-level spatial details to the decoder stage. The spatial attention module weights the encoded feature map to filter out redundant information at irrelevant positions and strengthen the spatial position information that is critical for generating accurate segmentation results. This improved jump connection mechanism can significantly enhance the model's ability to generate fine segmentation results and improve segmentation accuracy and robustness.

[0079] The structure of the spatial attention module is as follows Figure 4 As shown. The output feature map g of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are The input is to the spatial attention module. The output feature map of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are respectively passed through a 1×1 convolution layer, adjusted to the same number of feature channels C, and then added to obtain a fused feature map; the fused feature map is successively passed through the ReLU activation function, 1×1 convolution layer and normalization layer, and Sigmoid activation function to generate an attention weight map representing the importance of each spatial position. The attention weight map is multiplied by the input encoded feature map to obtain the output feature map of the spatial attention module.

[0080] The downsampling module is a ConvNeXt downsampling module, and its structure is as follows: Figure 5 As shown in the figure, the input features of the downsampling module are sequentially passed through the deep convolution layer, the normalization layer, the first convolution layer, the GELU activation function, and the second convolution layer to obtain the first downsampling features. The input features of the downsampling module are passed through the third convolution layer to obtain the second downsampling features. The first downsampling features and the second downsampling features are added to obtain the output features of the downsampling module.

[0081] Preferably, the depth convolution layer is a 3×3 depth convolution with a step size of 2; the first convolution layer, the second convolution layer and the third convolution layer are all 1×1 convolutions, and the step size of the third convolution layer is 2.

[0082] The upsampling module is a ConvNeXt upsampling module, and its structure is as follows: Figure 6 As shown in the figure, the input features of the upsampling module are sequentially passed through the first transposed convolution layer, the normalization layer, the fourth convolution layer, the GELU activation function, and the fifth convolution layer to obtain the first upsampling features. The input features of the upsampling module are passed through the second transposed convolution layer to obtain the second upsampling features. The first upsampling features and the second upsampling features are added to obtain the output features of the upsampling module.

[0083] Preferably, the first transposed convolution layer is a 3×3 transposed convolution with a step size of 2; the fourth convolution layer and the fifth convolution layer are both 1×1 convolution; and the second transposed convolution layer is a 1×1 transposed convolution with a step size of 2.

[0084] When training the intelligent image segmentation model, a deep supervision strategy is applied to the multi-scale output branches to optimize the training process and improve the model's ability to capture effective information at different scales. In addition to the segmentation prediction image finally output by the intelligent image segmentation model, additional supervision is performed on the outputs of the bottleneck layer and the first three stages of the decoder. The additional output branches can optimize the gradient transfer during back propagation, alleviate the gradient vanishing problem, and make the training process more stable. The output of the bottleneck layer and the output of each stage of the decoder are used as output branches. The predicted output of each output branch and the true segmentation mask are independently calculated for loss, and their weighted addition is used to obtain the total loss as the training loss. The formula is:

[0085] ;

[0086] in, is the total loss of the image intelligent segmentation model, n is the number of deep supervision output branches, and in this embodiment, n=5; is the weight coefficient of the i-th output branch, is the loss value of the i-th output branch. The sum of the weight coefficients of all output branches is 1, that is, .

[0087] This embodiment sets a larger weight for the high-level output branch, especially the final output branch, to emphasize its importance during training and to encourage the model to generate more refined segmentation results. During model training, a gradient clipping strategy is applied to constrain the gradient value to prevent gradient explosion and maintain the stability of the training process.

[0088] In summary, the present invention discloses a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, constructs an image intelligent segmentation model and applies the DConvNeXt module in the image intelligent segmentation model. The DConvNeXt module integrates the advanced components of Transformer, utilizes the dynamic feature aggregation characteristics of the DCNv4 operator and the inductive bias, integrates the advantages of the CNN and Transformer architectures, and effectively overcomes their respective limitations. Compared with conventional CNN methods, the DConvNeXt module can adaptively adjust the range of the receptive field according to the input, form an effective receptive field that matches the target area, and improve the shortcomings of CNN in modeling long-distance dependencies. Compared with the Transformer-based architecture, the DConvNeXt module is trained from scratch, does not rely on pre-training on large-scale data sets, and can converge faster with less training data. Therefore, the DConvNeXt module can better adapt to different image segmentation tasks and can be applied to a variety of scenarios.

[0089] The DConvNeXt module combines the deformable convolution v4 operator with dynamic feature modeling capabilities and the efficient local attention mechanism as a token mixer, which effectively improves the model's feature modeling capabilities while significantly enhancing the flexibility and dynamics of feature representation. Among them, the deformable convolution v4 operator dynamically adjusts the sampling position and sampling weight according to the input, and can flexibly deal with target areas with complex shapes. Combined with the efficient local attention mechanism, the model can further focus on key information and effectively capture the shape perception features of the target area.

[0090] In addition, the image intelligent segmentation model introduces a spatial attention module on the skip connection path, which can effectively optimize the features transmitted in the encoder stage and enhance key spatial position information. The present invention can effectively handle complex image segmentation tasks, significantly improves the adaptability in diversified application scenarios and the segmentation accuracy of complex images, and provides a novel and efficient solution for image segmentation services in different fields.

[0091] Embodiment 2

[0092] In order to verify the effect of the deep intelligent segmentation method based on inductive bias and dynamic feature aggregation proposed in the present invention, this embodiment applies it to the medical image segmentation task in the smart medical scenario and conducts experiments.

[0093] This example conducts experiments on three widely used and publicly available datasets, covering colon endoscopy polyp segmentation tasks and skin lesion segmentation tasks. The information of the datasets is shown in Table 1. The segmentation targets in these datasets are complex in morphology and highly irregular, which can effectively evaluate the performance of the model in complex image segmentation tasks.

[0094] Table 1. Dataset introduction

[0095] Dataset name type Segmentation target Sample size Image resolution Kvasir-SEG Colonoscopy images Polyp tissue 1000 Different sizes CVC-EndoSceneStill Colonoscopy images Polyp tissue 912 500×574，384×288 ISIC 2018 Dermoscopic images Lesion area 2594 Different sizes

[0096] The Kvasir-SEG dataset is a polyp segmentation dataset for computer-aided gastrointestinal disease detection, which aims to improve the quality of early detection of colorectal cancer. The dataset contains 1,000 polyp images of varying resolutions and corresponding segmentation masks, which are manually annotated by professional gastroenterologists. In this task, it is necessary to segment the morphologically diverse polyp tissue areas (label 1) and background areas (label 0) in the image. The experiment follows the widely used data preprocessing strategy in this task, randomly divides the dataset into training set, validation set, and test set in a ratio of 8:1:1, and uniformly adjusts the image size to 352×352.

[0097] CVC-EndoSceneStill Dataset is a colonoscopy image segmentation benchmark. The dataset is generated by combining two public colonoscopy image datasets, CVC-ColonDB and CVC-ClinicDB, and contains 912 polyp images and corresponding labeled images from 44 colonoscopy videos of 36 patients. The dataset owner provides a standard data partition, including 547 training images, 183 validation images, and 182 test images. The original data includes two image resolution sizes, 500 × 574 and 384 × 288. In order to facilitate subsequent processing, the image resolution is adjusted to 352×352.

[0098] ISIC 2018 is a skin lesion segmentation dataset provided by the International Skin Imaging Collaboration (ISIC). This dataset comes from the ISIC 2018 Challenge and includes high-resolution patient samples collected using a dermatoscope and segmentation masks reviewed by professional dermatologists. This example uses 2594 training samples as the dataset and randomly divides them into training set, validation set, and test set in a ratio of 8:1:1. Since the resolution of the original data is too high, nnUNet is used in the experiment to preprocess the images and resize the images to 256×256.

[0099] The following four evaluation indicators are used in the experiment to comprehensively evaluate the performance of the model in the image segmentation task: Mean Intersection over Union (mIoU), Dice Similarity Coefficient (Dice), Precision, and Recall.

[0100] mIoU and Dice are the most commonly used evaluation indicators in image segmentation. They both evaluate the overall quality of the segmentation result by measuring the degree of overlap between the predicted segmentation result and the true segmentation mask. The value range is 0 to 1, representing no overlap and complete overlap, respectively. The calculation formulas are expressed as:

[0101] ;

[0102] ;

[0103] in, Represents the predicted segmentation result, represents the true segmentation mask.

[0104] Precision and Recall are used as auxiliary evaluation indicators to evaluate the accuracy of segmentation results from different perspectives to achieve a more comprehensive evaluation of model performance. Precision represents the proportion of samples whose true labels are also positive among all samples whose predicted values ​​are positive, while Recall represents the proportion of samples that are correctly predicted among all positive samples. The calculation formula is:

[0105] ;

[0106] ;

[0107] Among them, TP, FP, and FN represent true positive, false positive, and false negative, respectively.

[0108] It is worth noting that in semantic segmentation, Dice is equivalent to the harmonic mean of Precision and Recall, that is, F1 Score. Using only Precision or Recall indicators is somewhat one-sided, and Dice can better show a comprehensive measure of the segmentation results.

[0109] The experimental environment of this embodiment is the basic condition for conducting experiments. The experiments in this article are conducted on a server with an Intel(R) Xeon(R) Platinum 8358P CPU @ 2.60GHz and a graphics card of NVIDIA RTX3090 (24GB). The experimental code is developed in Python and the model is implemented based on the Pytorch framework.

[0110] The parameters of the image intelligent segmentation model set in the experiment are shown in Table 2. Among them, the expansion ratio (R) represents the channel expansion ratio in the DConvNeXt module, and the number of groups (G) represents the number of groups of DCNv4 in the module. In order to better adapt to the medical image segmentation task, the architecture follows the implementation strategy of the standard U-Net architecture, setting the module number ratio of the four encoder stages to approximately 1:1:1:1, and using a symmetric design in the decoder stage. In order to improve computational efficiency, the channel expansion ratio of the first two encoder stages and the decoder stage is set to a smaller value, and the expansion ratio R=4 is set in the remaining stages to strike a balance between model performance and computational complexity. In the setting of the number of groups of DCNv4, the number of channels per group is kept at 16.

[0111] Table 2. Parameters of the image intelligent segmentation model

[0112] Parameter name Specifications Number of channels (C) Basic channel number C = 32 Number of modules (N) Ni=[3, 4, 4, 4, 4, 4, 4, 4, 3] Expansion ratio (R) Ri=[2, 3, 4, 4, 4, 4, 4, 3, 2] Number of groups (G) Gi=Ci / 16

[0113] The AdamW optimizer was used in the experiment, the initial learning rate was set to 0.0001, the batch size was 16, and 300 epochs were trained on each data set. In order to optimize the training effect of the model, the learning rate warm-up strategy was applied at the beginning to gradually increase the learning rate from a small value to the initial learning rate, and then the cosine annealing strategy was applied to gradually reduce the learning rate. The gradient checkpoint technology was used in training to save memory space. This method is a training strategy that trades time for space. It reduces the memory usage by recalculating some intermediate results of the forward propagation during the back-propagation process. The Albumenations library was used in the experiment to randomly apply horizontal and vertical flipping, translation, scaling, rotation, Gaussian noise, random elastic deformation, grid transformation and other data enhancement methods to the image to improve the model's adaptability to diverse input images.

[0114] This example evaluates the performance of the proposed intelligent image segmentation model on Kvasir-SEG, CVC-EndoSceneStill Dataset and ISIC 2018 datasets, and compares the results with a variety of state-of-the-art methods in recent years. The comparison algorithms include the classic image segmentation baseline model UNet, other advanced CNN methods (such as UNeXt, Rolling-Unet, DCRNet, DCSAU-Net, etc.), Transformer-based methods (TransFuse, FCBFormer, G-CASCADE), and ConvNeXt-based methods (MedNeXt).

[0115] Tables 3, 4, and 5 show the experimental results on the three datasets. The best results under each indicator are shown in bold, and the suboptimal results are underlined. The experimental results of the methods marked with "*" are taken from the original paper, and "—" indicates data not reported in the paper or not obtained in the experiment.

[0116] Table 3. Experimental results of Kvasir dataset (%)

[0117] Model Dice QUR Precision Recall UNet 89.38 81.11 91.24 87.83 TransFuse-S 90.74 85.34 — — UNeX 80.41 67.81 85.09 76.68 FCBFormer <![CDATA[ 92.19 ]]> <![CDATA[ 86.92 ]]> <![CDATA[ 93.30 ]]> 93.01 DCRNet* 90.14 84.44 93.10 90.35 MedNeXt 90.16 82.21 91.22 89.44 DUCK-Net 89.48 80.96 89.68 89.28 PVT G-CASCADE 91.41 86.39 90.46 <![CDATA[ 93.58 ]]> Rolling-Unet-L 90.73 83.29 91.77 89.94 The present invention 94.01 88.78 93.65 94.53

[0118] Table 4. Experimental results of CVC-EndoSceneStill dataset (%)

[0119] Model Dice QUR Precision Recall UNet 82.80 75.69 89.17 81.64 TransFuse-S 84.64 77.37 — — UNeX 62.88 48.13 81.92 54.35 DCRNet* 85.41 78.86 <![CDATA[ 91.74 ]]> 85.57 MedNeXt 82.57 75.10 87.91 84.12 Yin et al.* <![CDATA[ 86.03 ]]> <![CDATA[ 79.34 ]]> — — PVT G-CASCADE 85.84 79.18 90.64 86.94 Rolling-Unet-L 74.56 62.14 91.56 66.25 The present invention 86.31 80.04 92.26 <![CDATA[ 86.44 ]]>

[0120] Table 5. Experimental results of ISIC 2018 dataset (%)

[0121] Model Dice QUR Precision Recall UNet 91.30 84.23 93.36 89.56 TransFuse-S 91.69 85.78 — — UNeX 91.17 83.94 <![CDATA[ 93.19 ]]> 89.59 DCSAU-Net* 90.40 84.10 91.70 92.20 DuAT* <![CDATA[ 92.30 ]]> 86.70 — — MedNeXt 91.99 85.42 93.07 91.05 PVT G-CASCADE 91.94 85.87 90.59 93.73 Rolling-Unet-L 90.93 83.75 92.47 89.82 The present invention 92.69 <![CDATA[ 86.61 ]]> 92.20 <![CDATA[ 93.26 ]]>

[0122] The data shows that the present invention exhibits superior performance on the Kvasir-SEG and CVC-EndoSceneStill datasets, reaching the optimal level in almost all indicators, especially in the evaluation of the Dice indicator. On the ISIC2018 dataset, although the performance of the present invention in the individual indicators of Precision and Recall is not outstanding, it still achieves the highest score in the comprehensive evaluation indicator Dice, showing the best comprehensive performance. In addition, the experimental results of the present invention on the three datasets comprehensively surpass the baseline model UNet and the ConvNeXt-based model MedNeXt, which fully proves that the improved model architecture has more advantages in complex image segmentation tasks.

[0123] Experimental results show that the present invention can eliminate the interference of complex background in the image and accurately segment the target area with variable shapes.

[0124] In order to verify the effectiveness of different components in the image intelligent segmentation model proposed in the present invention and analyze the individual contribution of each component to the model performance, this embodiment conducts extensive ablation experiments on the Kvasir-SEG dataset, such as Figure 7 shown.

[0125] This embodiment first studies the influence of different token mixers in the DConvNeXt module on the model performance. The experimental results show that the method of the present invention (DCNv4+ELA) achieves the best performance. Compared with the model using 7×7 deep convolution, the model using only DCNv4 shows significant effects. After introducing the attention mechanism ELA, the model performance reaches the best. After that, this embodiment conducts an ablation study on the attention mechanism in the DConvNeXt module. The performance difference between ELA and several other advanced attention mechanisms is experimentally compared, including the channel attention SE module (squeeze-and-ExcitationModule), the mixed attention CBAM (Convolutional Block Attention Module), and CA (Coordinate Attention) similar to the ELA mechanism. The experimental results effectively verify the superior performance of ELA. In addition, this embodiment also ablates the influence of the skip connection mechanism and the deep supervision strategy. Removing the spatial attention module on the skip connection path or not enabling the deep supervision strategy will reduce the model performance.

[0126] In summary, the present invention can effectively handle complex image segmentation tasks, significantly improving the adaptability in diverse application scenarios and the segmentation accuracy of complex images.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A deep intelligent segmentation method based on inductive bias and dynamic feature aggregation, characterized in that: include: Build an intelligent image segmentation model, including the initial layer, encoder, bottleneck layer, decoder, spatial attention module, and output layer; Input the original image into the image intelligent segmentation model and output the segmentation prediction image, including: Input the original image into the initial layer of the image intelligent segmentation model to obtain the initial feature map; The initial feature map is input to the encoder to obtain the encoding feature map of each stage; the encoder includes four stages, each stage includes a DConvNeXt module and a downsampling module connected along the forward propagation direction; the output of the DConvNeXt module of each stage in the encoder is the encoding feature map of the stage; The DConvNeXt module includes a token mixer, a normalization layer, and a multilayer perceptron connected in sequence; the output features of the multilayer perceptron are added to the input features of the DConvNeXt module to obtain the output features of the DConvNeXt module; the token mixer includes a deformable convolution v4 operator and an efficient local attention mechanism connected in sequence, and the output features of the deformable convolution v4 operator are added to the output features of the efficient local attention mechanism to obtain the output features of the token mixer; The process of the DConvNeXt module is expressed as: ； ； ； in, represents the input features of the DConvNeXt module, represents the output features of the DConvNeXt module, represents the deformable convolution v4 operator, represents the output features of the deformable convolution v4 operator, represents an efficient local attention mechanism, represents the output characteristics of the token mixer, represents the normalization layer, represents a 1×1 convolution used to expand the number of channels from C to R times. represents the GELU activation function, Represents a 1×1 convolution used to restore the number of channels from CR to C; The deformable convolution v4 operator performs calculations on the input feature map through a sliding window. The calculation process is expressed as: ； in, Represents the output of the convolution operation on a sub-region of the input feature map of the deformable convolution v4 operator, Represents the sampling center point of the sub-region of the input feature map; Indicates the number of groupings of channels. Indicates Sub-feature graph corresponding to each group; Indicates The projection weights shared by the groups; Indicates the number of sampling points; Relative to the sampling center point No. The location coordinates of the sampling points; Indicates The position coordinates of the sampling points in the group The corresponding offset, Indicates In the group The modulation scalar of the sampling points; The calculation process of the efficient local attention mechanism is expressed as: ； ； ； in, represents the input feature map of the efficient local attention mechanism, represents the output feature map of the efficient local attention mechanism, and Represent the attention weights in the horizontal and vertical directions respectively, represents the Sigmoid activation function, represents group normalization, represents one-dimensional convolution, and Represents the average pooling operation in the horizontal and vertical directions respectively; Input the encoded feature map of the last stage in the encoder into the bottleneck layer to obtain the bottleneck layer output feature map; The output feature map of the bottleneck layer is input to the decoder to obtain a decoded feature map; the decoder includes four stages, each of which includes an upsampling module and a DConvNeXt module connected along the forward propagation direction; in the decoder, the output feature map of the upsampling module of each stage and the encoding feature map of the corresponding stage of the encoder are input to the spatial attention module through a jump connection, and the output feature map of the spatial attention module is added to the output feature map of the upsampling module of this stage as the input of the DConvNeXt module of this stage; the output of the DConvNeXt module of the last stage in the decoder is a decoding feature map; The decoded feature map is input into the output layer to obtain the segmentation prediction image.

2. According to claim 1, a deep intelligent segmentation method based on inductive bias and dynamic feature aggregation is characterized in that: The processing process of the spatial attention module includes: The output feature map of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are input into the spatial attention module. The output feature map of the decoder upsampling module and the encoded feature map of the corresponding stage of the encoder are respectively subjected to a 1×1 convolution layer, adjusted to the same number of feature channels C, and then added to obtain a fused feature map; the fused feature map is sequentially subjected to the ReLU activation function, 1×1 convolution layer and normalization layer, and Sigmoid activation function to generate an attention weight map representing the importance of each spatial position. The attention weight map is multiplied by the input encoded feature map to obtain the output feature map of the spatial attention module.

3. The deep intelligent segmentation method based on inductive bias and dynamic feature aggregation according to claim 1, characterized in that: The image intelligent segmentation model applies a deep supervision strategy to optimize the training process, and uses the output of the bottleneck layer and the output of each stage of the decoder as output branches to independently calculate the loss; the formula of the loss function is: ； in, is the total loss of the image intelligent segmentation model, and n is the number of deep supervision output branches; is the weight coefficient of the i-th output branch, is the loss value of the i-th output branch.

4. The deep intelligent segmentation method based on inductive bias and dynamic feature aggregation according to claim 1, characterized in that: The input features of the downsampling module are sequentially passed through a deep convolution layer, a normalization layer, a first convolution layer, a GELU activation function, and a second convolution layer to obtain a first downsampling feature. The input features of the downsampling module are passed through a third convolution layer to obtain a second downsampling feature. The output features of the downsampling module are obtained by adding the first downsampling feature and the second downsampling feature.

5. The deep intelligent segmentation method based on inductive bias and dynamic feature aggregation according to claim 1, characterized in that: The input features of the upsampling module are sequentially passed through the first transposed convolution layer, the normalization layer, the fourth convolution layer, the GELU activation function, and the fifth convolution layer to obtain the first upsampling features. The input features of the upsampling module are passed through the second transposed convolution layer to obtain the second upsampling features. The output features of the upsampling module are obtained by adding the first upsampling features and the second upsampling features.

Images