Evolution window attention adaptive system for image segmentation

Abstract

The invention discloses an evolutionary window attention self-adaption system for image segmentation. The evolutionary window attention self-adaption system comprises a super network module, an evolutionary architecture search module, a sub-network training module and an image segmentation execution module, the super network module is in communication connection with the evolutionary architecture search module, and the evolutionary architecture search module is in communication connection with the sub-network training module; the sub-network training module is in communication connection with the image segmentation execution module; and the super network module adopts an encoder-decoder architecture and comprises an encoder, a decoder and a multi-fusion hopping connection unit. According to the method, accurate segmentation of images with different modalities and different anatomical structures is realized, the accuracy, robustness and calculation efficiency of a segmentation result are improved, and clinical application requirements are met.

Description

Technical Field

[0001] This application relates to the field of image segmentation, and more specifically, to an evolutionary window attention adaptive system for image segmentation. Background Technology

[0002] Image segmentation is a fundamental task in the medical and health field. Accurately delineating anatomical structures and lesion areas in images is crucial for clinical decisions such as surgical planning and early disease diagnosis. For example, accurate segmentation of liver tumors in computed tomography (CT) images can provide a basis for surgical planning, while accurate segmentation of brain tissue in magnetic resonance imaging (MRI) images can assist in the early diagnosis of neurodegenerative diseases such as Alzheimer's disease by quantifying changes in gray and white matter structure.

[0003] Throughout the history of technological development, traditional methods based on Convolutional Neural Networks (CNNs), such as U-Net, U-Net++, and V-Net, have long held a dominant position in image segmentation due to their effective skip connections and detail recovery capabilities. However, these methods rely on local convolutional operations, which suffer from limited local receptive fields, making it difficult to effectively capture complex boundaries and long-range dependencies in images, easily leading to performance bottlenecks. Furthermore, expanding the receptive field by stacking more network layers introduces new problems such as increased computational complexity and vanishing gradients.

[0004] In recent years, Transformer models based on self-attention mechanisms, such as the Swin Transformer, have shown significant advantages in global information modeling. Their window attention mechanism effectively reduces computational complexity by dividing the input image into multiple fixed windows and performing self-attention computation within each window. The shifting window attention strategy can also enhance global information capture capabilities by enabling information interaction between non-adjacent windows through cyclic shifting. However, existing technologies still have the following key shortcomings: (1) Poor parameter adaptability: The window and shift size of models such as Swin Transformer are fixed values, which require manual adjustment and cannot adapt to the diverse anatomical structures, blurred boundaries and low contrast characteristics of images. For example, in images containing small blood vessels or tiny lesions, the fixed size makes it difficult to take into account both local details and global context, resulting in a decrease in segmentation accuracy; (2) Insufficient domain transfer: Existing Transformer backbone networks, such as Swing Transformer and Vision Transformer, are mainly designed for natural images and are not optimized for the high noise and highly specific features of images. Even if the weights are pre-trained using natural images through transfer learning, it is still difficult to fully capture the feature patterns unique to the medical field, and the domain bias problem is likely to occur; (3) Fixed upsampling strategy: The single upsampling method used in the decoding process cannot match the multi-scale feature extraction requirements of different images, such as CT, MRI and endoscopic images. For endoscopic images with blurred boundaries, the fixed upsampling strategy is difficult to accurately restore the spatial details of the lesion area, resulting in a large boundary segmentation error.

[0005] These shortcomings often lead to problems such as inaccurate boundaries, missing or oversegmenting small organs in image segmentation in existing methods, which cannot meet the clinical application requirements for high-precision segmentation results. Summary of the Invention

[0006] To address the aforementioned issues, this application provides an evolutionary window attention adaptive system for image segmentation, aiming to solve the technical problems of poor adaptability of window attention mechanisms, fixed upsampling strategies, and insufficient fusion of global and local features in existing image segmentation methods.

[0007] The first aspect of this invention provides an evolutionary window attention adaptive system for image segmentation, comprising: a supernetwork module, an evolutionary architecture search module, a subnetwork training module, and an image segmentation execution module; the supernetwork module is communicatively connected to the evolutionary architecture search module, the evolutionary architecture search module is communicatively connected to the subnetwork training module, and the subnetwork training module is communicatively connected to the image segmentation execution module; The hypernetwork module adopts an encoder-decoder architecture, wherein, The encoder includes multiple cascaded adaptive shift window transformer blocks, each transformer block being configured to perform a window shift operation based on a shift size selected from a predefined candidate set; The decoder includes multiple cascaded adaptive upsampling layers, each configured to select one from a variety of parallel upsampling units to improve the spatial resolution of features within the supernetwork module. A multi-fusion skip connection unit, connected between the encoder and the decoder, is configured to concatenate the output features of multiple different levels in the encoder and input them to the corresponding level of the decoder; The evolutionary architecture search module is configured to execute an evolutionary algorithm in a search space consisting of the shift size of the adaptive shift window transformer block and the upsampling strategy of the adaptive upsampling layer to select candidate sub-network architectures. The sub-network training module is configured to independently train the candidate sub-network architectures output by the evolutionary architecture search module, and determine the final image segmentation model based on the performance after full training. The image segmentation execution module is configured to perform segmentation operations on the input image using the final image segmentation model and output the segmentation results.

[0008] In one alternative implementation, the predefined candidate set is {0,1,2,3}.

[0009] In one optional implementation, the adaptive upsampling layer is provided with multiple upsampling units in parallel, including Patch Expand units, transposed convolution units, and CARAFE units.

[0010] In one optional implementation, the evolutionary architecture search module employs an asymmetric search strategy when executing the evolutionary algorithm; and performs constraint screening on the sub-network architecture based on preset upper limits for the number of parameters and computational complexity.

[0011] In one alternative implementation, the independent full training refers to the complete model training process starting from the initial state after resetting the weight parameters of the candidate sub-network architecture to the initial state.

[0012] A second aspect of this invention provides an evolutionary window attention adaptive method for image segmentation, comprising: A supernetwork is constructed based on an encoder-decoder architecture, and the supernetwork is trained; during the training process, By randomly sampling the shift size parameters of the adaptive shift window transformer block and the upsampling strategy parameters of the adaptive upsampling layer in the supernetwork, the shared weight parameters of the supernetwork are optimized to obtain the trained supernetwork. Based on the preset architecture search space, an evolutionary algorithm is used to search for the architecture of the trained supernetwork. The performance indicators of different architectures in the search space are evaluated using the shared weight parameters, and K candidate subnetwork architectures are selected. Each of the K candidate sub-network architectures is independently trained, and the optimal sub-network is selected as the final image segmentation model based on the performance metrics after complete training. The final image segmentation model is used to perform segmentation operations on the input image and output the segmentation results.

[0013] A third aspect of the present invention provides an electronic device, characterized in that it includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements an evolutionary window attention adaptive system for image segmentation.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, is used for an evolutionary window attention adaptive system for image segmentation.

[0015] This application has at least the following advantages or beneficial effects: (1) Strong adaptability and adaptability to multimodal images: The EW2A system of this application can achieve customized optimization for the characteristics of different modal images by dynamically matching the shift size and upsampling strategy through evolutionary search. In cardiac MRI (ACDC dataset), transposed convolution is selected as the main upsampling method to match clear myocardial boundaries; in abdominal CT (Synapse dataset), the CARAFE module is selected first to suppress interference caused by organ overlap; in esophageal endoscopy images (WLEC dataset), local lesion details are enhanced by small shift size.

[0016] (2) High segmentation accuracy meets clinical application needs: Compared with existing mainstream methods, the EW2A system of this application has made breakthroughs in key evaluation indicators, effectively solving the problem of low-contrast lesion segmentation. The high-precision segmentation results can provide a more reliable basis for clinical diagnosis and surgical planning, and reduce the subjective error of doctors.

[0017] (3) Excellent computational efficiency, balancing performance and practicality: The EW2A system of this application improves computational efficiency significantly while ensuring performance through a compact search space design and an efficient evolutionary algorithm.

[0018] (4) Robust and adaptable to complex clinical scenarios: strong noise resistance, the CARAFE module in the adaptive upsampling layer can suppress noise interference in the image through content awareness; accurate boundary segmentation, the multi-fusion skip connection enhances the expressive power of boundary features by aggregating multi-layer features; reliable small organ segmentation, the dynamic shift size can adjust the local attention range for small organs. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a framework diagram of an evolutionary window attention adaptive system for image segmentation proposed in one embodiment of this application; Figure 2 This is a flowchart of an evolutionary window attention adaptive method for image segmentation proposed in an embodiment of this application; Figure 3 This is a schematic diagram of an electronic device according to this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Please refer to Figure 1 , Figure 1 This is a framework diagram of an evolutionary window attention adaptive system for image segmentation proposed in one embodiment of this application. Figure 1 As shown in the figure, an embodiment of this application proposes an evolutionary window attention adaptive system for image segmentation, comprising: a supernetwork module 101, an evolutionary architecture search module 102, a subnetwork training module 103, and an image segmentation execution module 104; the supernetwork module 101 is communicatively connected to the evolutionary architecture search module 102, the evolutionary architecture search module 102 is communicatively connected to the subnetwork training module 103, and the subnetwork training module 103 is communicatively connected to the image segmentation execution module 104; The hypernetwork module 101 adopts an encoder-decoder architecture, wherein, The encoder includes multiple cascaded adaptive shift window transformer blocks, each transformer block being configured to perform a window shift operation based on a shift size selected from a predefined candidate set; In this embodiment, two consecutive adaptive shift window transformer blocks perform shift operations before calculating window attention. The shift size can be adaptively selected within the range of {0,1,2,3} (based on the 7×7 standard window size, this range can ensure the continuity of the window center pixel with the original context and avoid the loss of boundary information).

[0023] The specific steps of the module calculation process are as follows: Step 1, Input Features normalized layer After processing, it enters the adaptive shift window multi-head self-attention module. Adaptive shifted window multi-head self-attention module First, based on the selected displacement size Shift the window token, then perform multi-head self-attention computation, and the output is the same as the initial input. The updated features are obtained by adding the features through skip connections. The formula is as follows:

[0024] Step 2, Update Features After passing through the normalization layer again After processing, the input is given to a multilayer perceptron (MLP) for feature mapping. The output of the MLP is then compared with... The final output is obtained by adding the results through skip connections. The formula is as follows:

[0025] The self-attention calculation follows a standard formula:

[0026] Where Q, K, and V are the query, key, and value matrices, respectively (each with a dimension of 1). , (where d is the number of patches within the window, d is the query / key dimension), and B is the bias matrix. The derived attention bias.

[0027] The decoder includes multiple cascaded adaptive upsampling layers, each configured to select one from a variety of parallel upsampling units to improve the spatial resolution of features within the supernetwork module. A multi-fusion skip connection unit, connected between the encoder and the decoder, is configured to concatenate the output features of multiple different levels in the encoder and input them to the corresponding level of the decoder; In this embodiment, the multi-fusion skip connection unit is designed with a cross-layer feature aggregation mechanism to fuse multi-scale features from all previous layers of the encoder to the corresponding layer of the decoder. The specific steps are as follows: Step 1: Fusing features at the k-th layer of the decoder The output is obtained by the k-th layer adaptive shift window transformer of the decoder. and the output of the last VGG block of the encoder at layer k. The result obtained by piecing together: .

[0028] Step 2: Encoder features are progressively refined through hierarchical aggregation. For the output of the y-th VGG block in the x-th layer... (Saving y≤x), if y=1, it is obtained by concatenating the features of the (x-1)th layer and the xth layer of the encoder and then processing them with a VGG block; if y≥2, it is obtained by concatenating the output of the (y-1)th VGG block of the xth layer and the output of the (y-1)th VGG block of the (x-1)th layer and then processing them with a VGG block.

[0029] in, For the features of the encoder at layer x, The initial feature map is obtained by patching and embedding the input image; The VGG block uses the classic structure of "3×3 convolution + batch normalization (BN) + ReLU activation", as shown in the following formula: VGG(X)=ReLU(BN(Conv3×3(X))) Where X is the input feature tensor of the VGG block.

[0030] The evolutionary architecture search module 102 is configured to execute an evolutionary algorithm in a search space consisting of the shift size of the adaptive shift window transformer block and the upsampling strategy of the adaptive upsampling layer to select candidate sub-network architectures. In this embodiment, the evolutionary algorithm is based on the principles of natural selection and genetic variation. It iteratively optimizes and selects the optimal sub-network. The specific steps are as follows: Initialization: Generate an initial population of size 50 (randomly combine shift size and upsampling strategy), set the maximum number of iterations to 20, and the mutation probability to 0.15; Evaluation: Calculate the loss value of each individual in the population (sub-network) on the validation set (using a weighted sum of Dice loss and cross-entropy loss) as a performance evaluation metric; Selection: Based on the validation loss, the top 50% of individuals (25) were selected as parents to participate in subsequent breeding; Crossover: Randomly select 2 individuals from the parent generation as parents, and randomly crossover the shift size and upsampling strategy parameters (e.g., the upsampling index of parent generation 1 is [0,1,2,0], parent generation 2 is [1,0,1,2], and the offspring may be [0,0,2,2]), generating 25 crossover offspring; Mutation: Randomly mutate the parameters of the crossover offspring (each parameter has a 0.15 probability of being replaced with another candidate value, such as mutating the upsampled index [0,1,2,0] to [0,1,1,0]), generating 25 mutated offspring; Update: Merge crossover offspring and mutated offspring to form a new population (50 individuals), return to the "Evaluate" step for iterations until the maximum number of iterations is reached; Output: Select the 10 individuals with the smallest validation loss from the last generation population as candidate subnetworks.

[0031] The sub-network training module 103 is configured to independently train the candidate sub-network architectures output by the evolutionary architecture search module 102, and determine the final image segmentation model based on the performance after full training. The image segmentation execution module 104 is configured to perform segmentation operations on the input image using the final image segmentation model and output the segmentation results.

[0032] Furthermore, the predefined candidate set is {0,1,2,3}.

[0033] Furthermore, the adaptive upsampling layer is configured with multiple upsampling units in parallel, including Patch Expand units, transposed convolution units, and CARAFE units.

[0034] In this embodiment, adaptive feature recovery is achieved through dynamic selection to meet the feature recovery requirements of different images during the decoding process. During training, the upsampling selection module randomly activates a single module to ensure that all three methods are fully trained, providing diverse candidates for subsequent evolutionary search. The Patch Expand module directly improves the feature map resolution through patch expansion operations, preserving fine-grained spatial information, and is suitable for segmentation of large-sized anatomical structures; The transposed convolution module recovers the feature map size through inverse convolution, and has a strong boundary fitting ability, making it suitable for images with clear boundaries. The CARAFE module achieves upsampling based on content-aware feature rearrangement, which can adaptively suppress noise interference and is suitable for low-contrast, high-noise images.

[0035] Furthermore, when the evolutionary architecture search module executes the evolutionary algorithm, it adopts an asymmetric search strategy and performs constraint screening on the sub-network architecture based on preset upper limits for the number of parameters and the amount of computation.

[0036] In this embodiment, the total search space size = shift size space × upsampling strategy space; where... Shift size space: Candidate values ​​are {0,1,2,3}. The hypernet encoder and decoder each contain 3 layers. To maintain the symmetry of the U-shaped structure, the l-th layer of the encoder and the l-th layer of the decoder share the shift size. There are a total of 6 independent search layers, therefore the possible combinations of shift sizes are 4. 6 ; Upsampling strategy space: contains 3 candidate methods (Patch Expand, transposed convolution, CARAFE), and the decoder has a total of 4 upsampling layers, so the possible combinations of upsampling strategies are 34; Total search space size: 4 6 ×3 4 =331776, ensuring diversity while avoiding excessively long search times due to excessive space.

[0037] To ensure the model's practicality in clinical scenarios (such as adapting to mid-range medical devices), the following constraints are set: The number of parameters is constrained to ≤42M model parameters to avoid insufficient device memory. The computational constraint is ≤46 GMACs to ensure that the inference speed meets the real-time requirements of clinical practice.

[0038] Furthermore, the independent complete training refers to the complete model training process starting from the initial state after resetting the weight parameters of the candidate sub-network architecture to the initial state.

[0039] Please refer to Figure 2 , Figure 2 This is a flowchart of an evolutionary window attention adaptive method for image segmentation proposed in one embodiment of this application. Figure 2 As shown, this disclosure also provides an evolutionary window attention adaptive method for image segmentation, including: S100: Construct a supernetwork based on an encoder-decoder architecture and train the supernetwork; during the training process, S200: By randomly sampling the shift size parameters of the adaptive shift window transformer block and the upsampling strategy parameters of the adaptive upsampling layer in the supernetwork, the shared weight parameters of the supernetwork are optimized to obtain the trained supernetwork. S300: Based on the preset architecture search space, an evolutionary algorithm is used to search the architecture of the trained supernetwork. The shared weight parameters are used to evaluate the performance indicators of different architectures in the search space, and K candidate sub-network architectures are selected. S400: Perform independent full training on each of the K candidate sub-network architectures, and select the optimal sub-network as the final image segmentation model based on the performance metrics after full training. S500: Perform segmentation operation on the input image using the final image segmentation model and output the segmentation result.

[0040] This disclosure also provides an electronic device, please refer to... Figure 3 , Figure 3 This is a schematic diagram of an electronic device illustrated in an embodiment of this disclosure. For example... Figure 3 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus. The memory 110 stores a computer program that can run on the processor 120 to implement the steps of the evolutionary window attention adaptive system for image segmentation disclosed in this embodiment.

[0041] The disclosed embodiments also provide a computer-readable storage medium in which, when the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer device is enabled to perform steps as described in the embodiments of the present disclosure in an evolutionary window attention adaptive system for image segmentation.

[0042] In this embodiment, to verify the effectiveness of the Evolutionary Window Attention Adaptation (EW2A) system proposed in this invention, comprehensive experimental verification was conducted based on three typical image datasets with different characteristics. The specific implementation details are as follows: 1. Experimental Data Set and Evaluation Metrics This embodiment uses three publicly available image segmentation datasets to verify the generalization ability and effectiveness of the method of the present invention on different imaging modalities, different anatomical structures and different segmentation tasks.

[0043] The ACDC dataset contains 100 cardiac MRI images, requiring precise segmentation of three key structures: the left ventricle, right ventricle, and myocardium. In this embodiment, the dataset is divided into a training set of 70 images, a validation set of 10 images, and a test set of 20 images in a 7:1:2 ratio. Evaluation metrics used include the Dice coefficient (DSC) and the 95% Hausdorff distance (HD95), with DSC calculated individually for each structure.

[0044] Synapse Multi-Organ Segmentation Dataset: This dataset contains 30 abdominal CT images, with segmentation targets covering eight abdominal organs, including the liver, spleen, and pancreas. In this example, the images are divided into a training set (16 images), a validation set (2 images), and a test set (12 images) in a 16:2:12 ratio. The evaluation metrics are the mean DSC and mean HD95 for all organs.

[0045] The WLEC dataset contains 4740 endoscopic images of early esophageal cancer, characterized by challenging features such as low contrast and irregular boundaries in the lesion areas. In this embodiment, the images are divided into a training set of 3318 images, a validation set of 474 images, and a test set of 948 images in a 7:1:2 ratio. The evaluation metrics are DSC and HD95.

[0046] 2. Experimental Environment and Parameter Settings To ensure the reproducibility of the experiment and to explain the implementation process of the present invention in detail, the hardware and software environment and key training parameter settings used in this embodiment are as follows: Hardware environment: All training and testing were performed using an NVIDIA GeForce RTX 4090 GPU (24GB VRAM) to ensure efficient execution of large-batch training and complex calculations.

[0047] Software framework: Developed using the Python programming language and the PyTorch deep learning framework, fully leveraging its automatic differentiation and parallel computing capabilities to accelerate the model training process.

[0048] Initialization strategy: The weights of the supernetwork are initialized using weights pre-trained on the ImageNet dataset. This is intended to alleviate the overfitting problem that may be caused by the relative scarcity of image annotation data.

[0049] Data augmentation: During training, various data augmentation strategies were employed, including random flipping (horizontal / vertical), random rotation (-15° ~ +15°), and brightness and contrast adjustment, to expand the diversity of training data and improve the robustness of the model.

[0050] Training parameters: During the training phase of the supernet, the batch size was set to 32, and the training epochs were 350. During the subnetwork training phase, the batch size is also 32, and the training epochs are 150. The learning rate was initially set to 0.05 and decayed using a cosine annealing strategy. The optimizer used is stochastic gradient descent (SGD), with a momentum of 0.9 and a weight decay factor of 10. -4 ; Loss function: The weighted sum of Dice loss and cross-entropy loss is used as the optimization objective of the model, and its calculation formula is as follows:

[0051] Among them, Dice loss is mainly used to optimize the overlap between the segmented region and the ground truth label, while cross-entropy loss focuses on optimizing the classification accuracy at the pixel level. The combination of the two improves the overall performance of the segmentation model.

[0052] 3. Experimental Results Through experiments on the three datasets mentioned above, the EW2A method proposed in this invention achieved excellent segmentation performance.

[0053] Experimental results show that, compared with existing mainstream methods such as Swin-UNet and TransUNet, this method achieves significant improvements in DSC and HD95 metrics on the ACDC, Synapse, and WLEC datasets. It demonstrates significant advantages, especially in boundary segmentation accuracy, small organ segmentation performance, and adaptability to low-contrast images, fully verifying the effectiveness and advancement of this invention.

[0054] In summary, the ACDC dataset achieves the following DSC: 91.64% (a 1.64% improvement over Swin-UNet), HD95: 1.28 (a 0.06 decrease over Swin-UNet), and DSCs for the right ventricle, myocardium, and left ventricle are 90.20%, 88.87%, and 95.85%, respectively, all of which are currently state-of-the-art. Synapse dataset: Average DSC reaches 81.27% (0.19% improvement over VM-UNet), average HD95 is 17.83 (1.03 lower than CSWin-UNet), showing significant advantages in the segmentation of key organs such as liver and spleen; WLEC dataset: DSC reaches 69.01% (0.86% improvement over CSWin-UNet), HD95 is 36.50 (14.44 lower than TransUNet), effectively solving the problem of low-contrast lesion segmentation.

[0055] Search efficiency: The total training and search time is only 0.92 hours, which is 63% faster than ECT-NAS (2.49 hours) and 23% faster than AutoFormer (1.20 hours), reducing the model development cycle; Inference efficiency: The final model parameters are ≤42M, the computational cost is ≤46 GMACs, and the inference speed on a mid-range GPU can reach more than 15 frames per second, which meets the clinical real-time segmentation requirements; Training efficiency: The training method of sharing weights in the super network avoids the redundant computation of training a large number of sub-networks independently in the traditional neural architecture search, reducing training costs by more than 50%.

[0056] On the WLEC dataset, EW2A's HD95 is 14.44 lower than TransUNet, demonstrating its tolerance to noise. In myocardial boundary segmentation in the ACDC dataset, EW2A reduces the boundary error by more than 30% compared to UNet; On the Synapse dataset, EW2A achieves a DSC of 61.32% for the pancreas, a 4.74% improvement over Swin-UNet, thus solving the problem of missing segmentation of small organs in traditional methods.

[0057] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0058] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0060] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0061] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0062] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0063] The above provides a detailed description of an evolutionary window attention adaptive system for image segmentation provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An evolutionary window attention adaptive system for image segmentation, characterized in that, The super network module, the evolutionary architecture search module, the sub-network training module and the image segmentation execution module; the super network module is in communication connection with the evolutionary architecture search module, the evolutionary architecture search module is in communication connection with the sub-network training module; the sub-network training module is in communication connection with the image segmentation execution module; The super network module adopts an encoder-decoder architecture, wherein, The encoder comprises a plurality of cascaded adaptive shift window transformer blocks, each transformer block being configured to perform a window shift operation according to a shift size selected from a predefined candidate set; The decoder comprises a plurality of cascaded adaptive up-sampling layers, each up-sampling layer being configured to select one from a plurality of parallel up-sampling units to increase the spatial resolution of the internal features of the super network module; A multi-fusion skip connection unit is connected between the encoder and the decoder and is configured to splice the output features of a plurality of different levels in the encoder and input them to the corresponding levels of the decoder; The evolutionary architecture search module is configured to perform an evolutionary algorithm in a search space constituted by the shift sizes of the adaptive shift window transformer blocks and the up-sampling strategies of the adaptive up-sampling layers, and to screen candidate sub-network architectures; The sub-network training module is configured to independently and completely train the candidate sub-network architectures output by the evolutionary architecture search module, and to determine a final image segmentation model according to the performance after complete training; The image segmentation execution module is configured to perform a segmentation operation on an input image using the final image segmentation model and to output a segmentation result. The predefined candidate set is {0, 1, 2, 3}.

2. The evolutionary window attention adaptive system for image segmentation of claim 1, wherein, The adaptive up-sampling layer is provided with a plurality of up-sampling units in parallel, and the up-sampling units comprise a Patch Expand unit, a transpose convolution unit and a CARAFE unit.

3. The evolutionary window attention adaptive system for image segmentation of claim 1, wherein, When the evolutionary architecture search module performs the evolutionary algorithm, an asymmetric search strategy is adopted, and the sub-network architectures are constrained and screened based on a preset upper limit of the parameter quantity and a preset upper limit of the computation quantity.

4. The evolutionary window attention adaptive system for image segmentation of claim 1, wherein, The independent complete training refers to a complete model training process starting from an initial state after resetting the weight parameters of the candidate sub-network architectures to the initial state.

5. The evolutionary window attention adaptive system for image segmentation according to claim 1, wherein, The super network module, the evolutionary architecture search module, the sub-network training module and the image segmentation execution module; the super network module is in communication connection with the evolutionary architecture search module, the evolutionary architecture search module is in communication connection with the sub-network training module; the sub-network training module is in communication connection with the image segmentation execution module; 6. An evolutionary window attention adaptive method for image segmentation, characterized in that, The super network module adopts an encoder-decoder architecture, wherein, The encoder comprises a plurality of cascaded adaptive shift window transformer blocks, each transformer block being configured to perform a window shift operation according to a shift size selected from a predefined candidate set; The decoder comprises a plurality of cascaded adaptive up-sampling layers, each up-sampling layer being configured to select one from a plurality of parallel up-sampling units to increase the spatial resolution of the internal features of the super network module; A multi-fusion skip connection unit is connected between the encoder and the decoder and is configured to splice the output features of a plurality of different levels in the encoder and input them to the corresponding levels of the decoder; The evolutionary architecture search module is configured to perform an evolutionary algorithm in a search space constituted by the shift sizes of the adaptive shift window transformer blocks and the up-sampling strategies of the adaptive up-sampling layers, and to screen candidate sub-network architectures; The sub-network training module is configured to independently and completely train the candidate sub-network architectures output by the evolutionary architecture search module, and to determine a final image segmentation model according to the performance after complete training; The image segmentation execution module is configured to perform a segmentation operation on an input image using the final image segmentation model and to output a segmentation result.

7. An electronic device, comprising: The predefined candidate set is {0, 1, 2, 3}. The adaptive up-sampling layer is provided with a plurality of up-sampling units in parallel, and the up-sampling units comprise a Patch Expand unit, a transpose convolution unit and a CARAFE unit. When the evolutionary architecture search module performs the evolutionary algorithm, an asymmetric search strategy is adopted, and the sub-network architectures are constrained and screened based on a preset upper limit of the parameter quantity and a preset upper limit of the computation quantity. The independent complete training refers to a complete model training process starting from an initial state after resetting the weight parameters of the candidate sub-network architectures to the initial state. The super network module, the evolutionary architecture search module, the sub-network training module and the image segmentation execution module; the super network module is in communication connection with the evolutionary architecture search module, the evolutionary architecture search module is in communication connection with the sub-network training module; the sub-network training module is in communication connection with the image segmentation execution module; A processor, a memory, and a computer program stored on the memory and executable on the processor, the processor implementing the evolutionary window attention adaptive system for image segmentation of any one of claims 1 to 5 when executing the program.

8. A computer-readable storage medium, characterized in that, A computer program stored on the computer readable storage medium, the computer program implementing the evolutionary window attention adaptive system for image segmentation of any one of claims 1 to 5 when executed by a processor.

Images