Tumor image segmentation method and device, electronic equipment and storage medium

Abstract

The invention provides a tumor image segmentation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target tumor image, processing the target tumor image through employing a tumor image segmentation model, and obtaining a target tumor image segmentation result; the training of the tumor image segmentation model comprises the following steps: dividing a tumor image in a data set into image blocks; performing down-sampling processing on the image block to obtain a first feature map; performing up-sampling processing on the first feature map, performing multi-head self-attention calculation by adopting a cascaded Swin Transform to obtain a second feature map, and performing cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; and performing segmentation effect evaluation by adopting cross entropy loss and Dice loss to obtain a tumor image segmentation model. The method has the beneficial effects that the segmentation efficiency and precision are improved, and the system resource consumption during segmentation is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a tumor image segmentation method, device, electronic equipment and storage medium. Background Art

[0002] With advances in deep learning technology, the field of medical image segmentation, particularly for tumor-related images, has also seen significant progress. Medical image segmentation technology is crucial in clinical practice, helping doctors accurately identify tumor regions and their boundaries within complex MRI, CT, ultrasound, and PET images. This technology is crucial for tumor diagnosis, treatment planning, and efficacy evaluation.

[0003] Many models are expensive to train, requiring extensive computing resources and time, which limits their application. Furthermore, these models often require specialized network architectures and parameter adjustments tailored to specific tasks, lacking universality and flexibility. Furthermore, early diagnosis of tumors relies on accurate analysis of medical images, but traditional image analysis methods often rely on physician experience, which is highly subjective and time-consuming. Furthermore, tumor morphology, size, location, and contrast vary significantly between patients, making traditional segmentation methods difficult to universally apply.

[0004] The rapid development of deep learning technology has provided new solutions for medical image segmentation. In particular, models based on architectures such as U-Net and Transformer have achieved significant progress in various medical imaging tasks. However, current large-scale deep learning models are still dominated by disease-specific models. Each model is typically trained for a specific type of tumor (such as lung cancer, glioma, or liver cancer), making it difficult to generalize to different tumor types or multimodal data. Even general models developed in recent years, such as MedSAM (the Medical Segmentation Self-Attention Model) and Med-CLIP, trained in a medical context, still require fine-tuning for specific tasks to achieve high segmentation accuracy and are essentially extensions of disease-specific models.

[0005] Although deep learning has shown great potential in medical image segmentation, existing methods still face problems such as high training cost, poor model generalization ability and strong task dependence, which can lead to inaccurate segmentation of tumor-related images (especially small objects) and inability to cope with the segmentation of different tumor types. Summary of the Invention

[0006] The main purpose of the embodiments of the present invention is to provide a tumor image segmentation method, device, electronic device and storage medium, which improve the segmentation efficiency and accuracy and reduce the system resource consumption during segmentation.

[0007] One aspect of the present invention provides a tumor image segmentation method, comprising: Acquire a target tumor image, and process the target tumor image using a tumor image segmentation model to obtain a target tumor image segmentation result; The training of the tumor image segmentation model includes: Divide the tumor images in the dataset into image blocks; Performing downsampling processing on the image block to obtain a first feature map; Performing upsampling processing on the first feature map, and performing multi-head self-attention calculation using a cascaded Swin Transformer to obtain a second feature map, and performing cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; The segmentation effect of the third feature map is evaluated using cross entropy loss and Dice loss to obtain the tumor image segmentation model.

[0008] According to the tumor image segmentation method, performing downsampling processing on the image block to obtain a first feature map includes: The image block is downsampled by using Patch Merging through a four-level encoder to obtain the first feature map; wherein, when each level of the encoder performs downsampling, the resolution of the image block is reduced to 1 / 2, and a learnable linear embedding layer is used to expand the channel dimension to 2 times.

[0009] According to the tumor image segmentation method, performing upsampling processing on the first feature map to obtain a second feature map includes: Progressive Patch Expanding is used for upsampling through a four-level decoder. Patch Expanding uses a hybrid upsampling strategy of transposed convolution and bilinear interpolation to perform upsampling. Patch Expanding includes parallel local attention branches and global context branches. The local attention branch uses 7×7 large kernel convolution to capture local details, and the global context branch uses non-local attention to establish long-range dependencies.

[0010] According to the tumor image segmentation method, a cascaded Swin Transformer is used to perform multi-head self-attention calculation, and a cascaded Swin Transformer is used to perform multi-head self-attention calculation, including: The Swin Transformer based on sliding window is used to perform multi-head self-attention calculation to obtain the self-attention calculation result, where the Swin Transformer includes a LayerNorm layer, a residual connection block and a two-layer GELU nonlinear fully connected network, and any connected Swin Transformer includes a window-based multi-head self-attention module and a sliding window-based multi-head self-attention module; Among them, the LayerNorm layer performs batch normalization on the first feature map to obtain a normalized result; The normalized results are subjected to multi-head self-attention calculation through the cascaded Swin Transformer, where the processing formula of the connected Swin Transformer is: in, and Respectively represent the outputs of the multi-head self-attention module and the MLP module in the l-th Swin Transformer, W-MSA represents the window-based multi-head self-attention module, and SW-MSA represents the sliding window-based multi-head self-attention module; The self-attention calculation formula is: , in, 、 、 represent the query matrix, key matrix, and value matrix respectively; is the number of image blocks in each window, represents the dimension of the query / key in each window, B is the bias term, and the bias matrix Sure, The dimension is According to the tumor image segmentation method, cross-modal feature fusion is performed on the first feature map and the second feature map to obtain a third feature map, including: The local entropy values ​​of the first feature map and the second feature map are calculated by RSU4F using a dynamic void rate selection algorithm, and the expansion coefficient of the parallel convolution branch of the deep learning model is automatically adjusted according to the local entropy values. The deep learning model adopts Swin-U2Net; Through the pyramid consistency constraint loss function, the prediction results of different scales can meet the requirements in the feature space. Distance minimization, and, through adversarial training, minimizing the Wasserstein distance between the predicted marginal distribution and the true annotation; The mutual information of the first feature map and the second feature map is fused using cross-attention features, a modal weight matrix is ​​dynamically generated, and soft attention weighted fusion is performed in the channel dimension through the modal weight matrix to obtain the third feature map.

[0011] According to the tumor image segmentation method, the training of the tumor image segmentation model further includes: A three-layer gradient scaling strategy is used for mixed precision training, where the three-layer gradient scaling strategy includes a backbone network using Float16, an attention module using BrainFloat16, and loss calculation using Float32, and training is performed through a dynamic gradient clipping algorithm; a feature map cache compression algorithm is used to compress the tumor image segmentation model; and GroupNorm and deep convolution are used to accelerate inference of the tumor image segmentation model training.

[0012] According to the tumor image segmentation method, the segmentation effect of the third feature map is evaluated using cross entropy loss and Dice loss, including: Cross entropy loss is used to evaluate the pixel-level segmentation effect, where cross entropy loss includes weighted cross entropy, multi-scale processing and smoothing processing; The Dice loss is used to evaluate the overlap of the segmented regions, where the Dice loss includes the mean Dice coefficient, the correlation Dice coefficient, and the narrow area Dice coefficient. Used to measure the similarity between the predicted result P and the foreground area of ​​the true label G, related Dice coefficient By expanding the irrelevant area of ​​the intersection of the true label G and the prediction result Exclude, narrow area Dice coefficient Pre-labeled narrow labels The intersection evaluation prediction result with the true label is: The continuity and integrity of the extracted blood vessel centerline were evaluated using the error front overlap and overlap degree, and the Hausdorff distance was used to measure the similarity between two point sets to evaluate the segmentation effect.

[0013] Another aspect of an embodiment of the present invention provides a tumor image segmentation device, comprising: The first module is used to obtain a target tumor image, and process the target tumor image using a tumor image segmentation model to obtain a target tumor image segmentation result; The training of the tumor image segmentation model includes: The second module is used to divide the tumor images in the dataset into image blocks; A third module is configured to perform downsampling processing on the image block to obtain a first feature map; A fourth module is configured to perform upsampling processing on the first feature map, and perform multi-head self-attention calculation using a cascaded SwinTransformer to obtain a second feature map, and perform cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; The fifth module is used to evaluate the segmentation effect of the third feature map using cross entropy loss and Dice loss to obtain the tumor image segmentation model.

[0014] Another aspect of an embodiment of the present invention provides an electronic device, including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method described above.

[0015] An embodiment of the present invention further discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions stored in a computer-readable storage medium.

[0016] The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described above.

[0017] The beneficial effects of the present invention are as follows: the self-attention mechanism based on local windows can effectively capture the global information and long-distance dependencies in the image, and combined with the deep supervision and nested connection structure of U2Net, it improves the detail recovery ability and segmentation accuracy, and can extract network features from more resolutions. While changing the resolution, the number of channels is increased to avoid information loss; using Patch Merging and Patch Expand for downsampling and upsampling, the model performs deep multi-scale feature fusion in the encoder and decoder, which can effectively process feature maps of different resolutions, helping the model to provide high-precision segmentation results when the tumor morphology, size and position vary greatly; using the RSU4F module, through a fully parallel void convolution structure, using convolutions with different expansion rates to extract multi-scale contextual information in parallel, it can effectively expand the receptive field and retain spatial resolution, which is particularly suitable for boundary clarity and small target detection in tumor segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which: Figure 1 4 is a schematic diagram of the tumor image segmentation process according to an embodiment of the present invention.

[0019] Figure 2 4 is a network structure diagram of Swin-U2Net according to an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of a continuous Swin Transformer module according to an embodiment of the present invention.

[0021] Figure 4 4 is a schematic diagram of the multi-scale feature fusion process of an embodiment of the present invention.

[0022] Figure 5 FIG. 4 is a schematic diagram of a tumor image segmentation device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. In the subsequent description, suffixes such as "module," "component," or "unit" used to represent elements are used solely to facilitate the description of the present invention and have no specific meaning in themselves. Therefore, "module," "component," or "unit" may be used interchangeably. "First," "second," and the like are used solely to distinguish technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In this subsequent description, the consecutive numbering of method steps is for ease of review and understanding. In conjunction with the overall technical solution of the present invention and the logical relationship between the various steps, adjusting the order of implementation of the steps does not affect the technical effects achieved by the technical solution of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and should not be construed as limiting the present invention.

[0024] refer to Figure 1 , Figure 1 FIG. 1 is a schematic diagram of tumor image segmentation according to an embodiment of the present invention, which includes but is not limited to steps S100 to S500: S100 , acquiring a target tumor image, and processing the target tumor image using a tumor image segmentation model to obtain a target tumor image segmentation result.

[0025] In some embodiments, reference Figure 2The network structure diagram of a tumor image segmentation model shown is Swin-U2Net, which includes: (1) A five-stage encoder, (2) a four-stage decoder, and (3) a saliency map fusion module connected to the decoder stage and the final encoder stage.

[0026] The Swin-U2Net model combines the advantages of Swin Transformer and U2Net, adopts a local window-based self-attention mechanism, can effectively capture the global information and long-distance dependencies in the image, and combines the deep supervision and nested connection structure of U2Net to improve detail recovery capabilities and segmentation accuracy.

[0027] Patch Merging and Patch Expand are used for downsampling and upsampling respectively. The model performs deep multi-scale feature fusion in the encoder and decoder, which can effectively process feature maps of different resolutions, helping the model to provide high-precision segmentation results when tumor morphology, size, and location vary greatly.

[0028] During segmentation, Patch In divides the input image into smaller patches, converting the original image into a set of smaller feature representations. This significantly reduces the spatial dimension of the image and, in turn, the complexity of subsequent computations. Compared to the pooling layer, Patch Merging aggregates each patch when segmenting the image, enabling the model to retain more detailed features and avoiding the information loss caused by a fixed window.

[0029] The RSU4F module uses a fully parallel atrous convolution structure and convolutions with different expansion rates to extract multi-scale contextual information in parallel, which can effectively expand the receptive field and retain spatial resolution. It is particularly suitable for boundary clarity and small target detection in tumor segmentation.

[0030] The training of the tumor image segmentation model includes: S200, dividing the tumor image in the dataset into image blocks.

[0031] In some embodiments, combined Figure 2 , which divides the input image into small Patches (image blocks) by using the Patch in module.

[0032] S300: Perform downsampling processing on the image block to obtain a first feature map.

[0033] It can be understood that the first feature map is a feature map obtained by downsampling the encoder.

[0034] In some embodiments, during the encoding phase, the model employs four levels of patch merging downsampling, reducing the feature map resolution to 1 / 2 at each level. Simultaneously, a learnable linear embedding layer expands the channel dimension by a factor of 2, forming a pyramidal feature representation. This model introduces a cross-stage feature reorganization mechanism based on the skip connections of traditional U-Net, addressing the semantic gap when fusing low-level and high-level features.

[0035] S400, performing upsampling processing on the first feature map, and using a cascaded Swin Transformer to perform multi-head self-attention calculation to obtain a second feature map, and performing cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map.

[0036] In some embodiments, the decoder employs progressive patch expanding upsampling, an innovative hybrid upsampling strategy that combines transposed convolution with bilinear interpolation. Each upsampling module includes a parallel local attention branch and a global context branch. The former captures local details through a large 7×7 kernel convolution, while the latter uses a lightweight non-local attention module to establish long-range dependencies.

[0037] In some embodiments, the decoder further includes a Swin Transformer module, referring to Figure 3 The diagram of the continuous Swin Transformer module shown in Figure 1 shows that each Swin Transformer module includes a LayerNorm (LN) layer, a multi-head self-attention module (MSA), a residual connection, and a two-layer MLP (fully connected network) with GELU nonlinearity. The formula for the multi-head attention calculation of the continuous Swin Transformer module is:

[0038] In these two consecutive Transformer modules, the window-based multi-head self-attention (W-MSA) module and the sliding window-based multi-head self-attention (SW-MSA) module are used respectively. Based on this window division mechanism, the consecutive Swin Transformer modules can be formally expressed as: in, and They represent the outputs of the SW-MSA module or the W-MSA and MLP modules in the lth module respectively. W-MSA represents the window-based multi-head self-attention module, and SW-MSA represents the sliding window-based multi-head self-attention module.

[0039] The self-attention of the embodiment of the present invention is calculated as follows: , in 、 、 Represent query, key, and value matrices respectively; and Represents the number of patches in each window and the dimension of the query / key, is the dot product of the query vector and the key vector.

[0040] B is the bias term, and the value in B is taken from the bias matrix , bias matrix The dimension is .

[0041] In some embodiments, reference Figure 4 The multi-scale feature fusion process diagram shown in FIG4 includes but is not limited to steps S410 to S430: S410, calculating the local entropy values ​​of the first feature map and the second feature map using a dynamic void rate selection algorithm through RSU4F, and automatically adjusting the expansion coefficient of the parallel convolution branch of the deep learning model according to the local entropy values, wherein the deep learning model adopts Swin-U2Net; S420, through the pyramid consistency constraint loss function, the prediction results of different scales are minimized in the feature space to meet the L2 distance, and the Wasserstein distance between the predicted marginal distribution and the true annotation is minimized through adversarial training; S430, using cross-attention features to fuse the mutual information of the first feature map and the second feature map, dynamically generating a modal weight matrix, and performing soft attention weighted fusion in the channel dimension through the modal weight matrix to obtain a third feature map.

[0042] In some embodiments, the training design of the tumor image segmentation model includes: Hybrid precision uses a three-level gradient scaling strategy, including using Float16 for the backbone network, BrainFloat16 for the attention module, and Float32 for loss calculation. Combined with a dynamic gradient clipping algorithm (the threshold decays cosine with the training cycle), it enables large-scale training (>1M images) with a batch size of up to 1024 without losing accuracy.

[0043] In some embodiments, the loss_ce loss weight is set to 0.4, the loss_dice loss weight is set to 0.6, and loss = loss_ce 0.4+loss_dice 0.6, total_loss = sum(loss), calculate the loss of the output list of the third feature map and add it up to the total loss, including: Dice dominance (0.6) ensures that the model pays attention to the overall structure of the target region and the class imbalance problem.

[0044] CE-assisted (0.4), provides a stable gradient signal to supplement pixel-level classification accuracy.

[0045] Memory optimization uses the Feature Cache Compression (FCC) algorithm and tensor Tucker decomposition to reduce intermediate feature storage requirements by 70% while maintaining 95% of feature energy. This enables the model to process 512×512×512 volumetric data on consumer-grade GPUs such as the RTX3090.

[0046] To accelerate inference, replace LayerNorm in the Swin Transformer block with GroupNorm and use depthwise convolution to replace some matrix multiplication operations.

[0047] S500: Cross entropy loss and Dice loss are used to evaluate the segmentation effect of the third feature map to obtain a tumor image segmentation model.

[0048] In some embodiments, cross entropy loss is used to evaluate the pixel-level segmentation effect, wherein the cross entropy loss includes weighted cross entropy, multi-scale processing and smoothing processing; The Dice loss is used to evaluate the overlap of the segmented regions, where the Dice loss includes the mean Dice coefficient, the correlation Dice coefficient, and the narrow area Dice coefficient. Used to measure the similarity between the predicted result P and the foreground area of ​​the true label G, related Dice coefficient By expanding the irrelevant area of ​​the intersection of the true label G and the prediction result Exclude, narrow area Dice coefficient Pre-labeled narrow labels The intersection evaluation prediction result with the true label is: The continuity and integrity of the extracted blood vessel centerline were evaluated using the pre-error overlap and overlap degree, and the Hausdorff distance was used to measure the similarity between two point sets.

[0049] Figure 5 FIG. 5 is a diagram of a tumor image segmentation and analysis device according to an embodiment of the present invention. The device includes a first module 510 , a second module 520 , a third module 530 , a fourth module 540 , and a fifth module 550 .

[0050] Among them, the first module is used to obtain the target tumor image, and process the target tumor image using the tumor image segmentation model to obtain the target tumor image segmentation result; the training of the tumor image segmentation model includes: the second module is used to divide the tumor images in the data set into image blocks; the third module is used to perform downsampling processing on the image blocks to obtain the first feature map; the fourth module is used to perform upsampling processing on the first feature map, and use the cascaded Swin Transformer to perform multi-head self-attention calculation to obtain the second feature map, and perform cross-modal feature fusion on the first feature map and the second feature map to obtain the third feature map; the fifth module is used to use cross entropy loss and Dice loss to evaluate the segmentation effect of the third feature map to obtain the tumor image segmentation model.

[0051] Exemplarily, with the cooperation of the first to fifth modules in the device, the embodiment device can implement any of the aforementioned tumor image segmentation methods, that is, obtaining a target tumor image, processing the target tumor image using a tumor image segmentation model, and obtaining a target tumor image segmentation result; the training of the tumor image segmentation model includes: dividing the tumor image in the data set into image blocks; performing downsampling processing on the image blocks to obtain a first feature map; performing upsampling processing on the first feature map, and using a cascaded Swin Transformer to perform multi-head self-attention calculation to obtain a second feature map, performing cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; using cross entropy loss and Dice loss to evaluate the segmentation effect of the third feature map to obtain a tumor image segmentation model. The beneficial effects of the present invention are as follows: the self-attention mechanism based on local windows can effectively capture the global information and long-distance dependencies in the image, and combined with the deep supervision and nested connection structure of U2Net, it improves the detail recovery ability and segmentation accuracy, and can extract network features from more resolutions. While changing the resolution, the number of channels is increased to avoid information loss; using PatchMerging and Patch Expand for downsampling and upsampling, the model performs deep multi-scale feature fusion in the encoder and decoder, which can effectively process feature maps of different resolutions, helping the model to provide high-precision segmentation results when the tumor morphology, size and position vary greatly; the RSU4F module uses a fully parallel void convolution structure and convolutions with different expansion rates to extract multi-scale contextual information in parallel, which can effectively expand the receptive field and retain spatial resolution, and is particularly suitable for boundary clarity and small target detection in tumor segmentation.

[0052] An embodiment of the present invention further provides an electronic device, the electronic device including a processor and a memory; The memory stores a program; The processor executes the program to perform the aforementioned tumor image segmentation method; the electronic device has the function of carrying and running the software system for tumor image segmentation provided by the embodiment of the present invention, for example, a personal computer, a minicomputer, a main frame, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or communicates with a charged particle tool or other imaging device, etc.

[0053] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the tumor image segmentation method as described above.

[0054] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0055] Embodiments of the present invention further disclose a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned tumor image segmentation method.

[0056] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0057] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0058] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0059] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0060] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0061] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0062] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0063] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A tumor image segmentation method, characterized in that: include: Acquire a target tumor image, and process the target tumor image using a tumor image segmentation model to obtain a target tumor image segmentation result; The training of the tumor image segmentation model includes: Divide the tumor images in the dataset into image blocks; Performing downsampling processing on the image block to obtain a first feature map; Performing upsampling processing on the first feature map, and performing multi-head self-attention calculation using a cascaded Swin Transformer to obtain a second feature map, and performing cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; The segmentation effect of the third feature map is evaluated using cross entropy loss and Dice loss to obtain the tumor image segmentation model.

2. The tumor image segmentation method according to claim 1, characterized in that: The performing downsampling processing on the image block to obtain a first feature map includes: The image block is downsampled by using Patch Merging through a four-level encoder to obtain the first feature map; wherein, when each level of the encoder performs downsampling, the resolution of the image block is reduced to 1 / 2, and a learnable linear embedding layer is used to expand the channel dimension to 2 times.

3. The tumor image segmentation method according to claim 1, characterized in that: The upsampling process is performed on the first feature map, and the multi-head self-attention calculation is performed using a cascaded Swin Transformer to obtain a second feature map, including: Progressive Patch Expanding is used for upsampling through a four-level decoder. Patch Expanding uses a hybrid upsampling strategy of transposed convolution and bilinear interpolation to perform upsampling. Patch Expanding includes parallel local attention branches and global context branches. The local attention branch uses 7×7 large kernel convolution to capture local details, and the global context branch uses non-local attention to establish long-range dependencies.

4. The tumor image segmentation method according to claim 1, characterized in that: The multi-head self-attention calculation using cascaded SwinTransformer includes: The Swin Transformer based on sliding window is used to perform multi-head self-attention calculation to obtain the self-attention calculation result, where the Swin Transformer includes a LayerNorm layer, a residual connection block and a two-layer GELU nonlinear fully connected network, and any connected Swin Transformer includes a window-based multi-head self-attention module and a sliding window-based multi-head self-attention module; Among them, the LayerNorm layer performs batch normalization on the first feature map to obtain a normalized result; The normalized results are subjected to multi-head self-attention calculation through the cascaded Swin Transformer, where the processing formula of the connected Swin Transformer is: in, and Respectively represent the outputs of the multi-head self-attention module and the MLP module in the l-th Swin Transformer, W-MSA represents the window-based multi-head self-attention module, and SW-MSA represents the sliding window-based multi-head self-attention module; The self-attention calculation formula is: ， in, 、 、 , represent the query matrix, key matrix and value matrix respectively; is the number of image blocks in each window, represents the dimension of the query / key in each window, B is the bias term, and the bias matrix Sure, The dimension is .

5. The tumor image segmentation method according to claim 1, characterized in that: The cross-modal feature fusion of the first feature map and the second feature map to obtain a third feature map includes: The local entropy values ​​of the first feature map and the second feature map are calculated by RSU4F using a dynamic void rate selection algorithm, and the expansion coefficient of the parallel convolution branch of the deep learning model is automatically adjusted according to the local entropy values. The deep learning model adopts Swin-U2Net; Through the pyramid consistency constraint loss function, the prediction results of different scales can meet the requirements in the feature space. Distance minimization, and, through adversarial training, minimizing the Wasserstein distance between the predicted marginal distribution and the true annotation; The mutual information of the first feature map and the second feature map is fused using cross-attention features, a modal weight matrix is ​​dynamically generated, and soft attention weighted fusion is performed in the channel dimension through the modal weight matrix to obtain the third feature map.

6. The tumor image segmentation method according to claim 1, characterized in that: The training of the tumor image segmentation model further includes: A three-layer gradient scaling strategy is used for mixed precision training, where the three-layer gradient scaling strategy includes a backbone network using Float16, an attention module using BrainFloat16, and loss calculation using Float32, and training is performed through a dynamic gradient clipping algorithm; a feature map cache compression algorithm is used to compress the tumor image segmentation model; and GroupNorm and deep convolution are used to accelerate inference of the tumor image segmentation model training.

7. The tumor image segmentation method according to claim 1, characterized in that: The segmentation effect evaluation of the third feature map using cross entropy loss and Dice loss includes: Cross entropy loss is used to evaluate the pixel-level segmentation effect, where cross entropy loss includes weighted cross entropy, multi-scale processing and smoothing processing; The Dice loss is used to evaluate the overlap of the segmented regions, where the Dice loss includes the mean Dice coefficient, the correlation Dice coefficient, and the narrow area Dice coefficient. Used to measure the similarity between the predicted result P and the foreground area of ​​the true label G, related Dice coefficient By expanding the irrelevant area of ​​the intersection of the true label G and the prediction result Exclude, narrow area Dice coefficient Pre-labeled narrow labels The intersection evaluation prediction result with the true label is: The continuity and integrity of the extracted vascular centerline were evaluated using the error front overlap and overlap degree, and the Hausdorff distance was used to measure the similarity between two point sets to evaluate the segmentation effect.

8. A tumor image segmentation device, characterized in that: include: The first module is used to obtain a target tumor image, and process the target tumor image using a tumor image segmentation model to obtain a target tumor image segmentation result; The training of the tumor image segmentation model includes: The second module is used to divide the tumor images in the dataset into image blocks; A third module is configured to perform downsampling processing on the image block to obtain a first feature map; A fourth module is configured to perform upsampling processing on the first feature map, and perform multi-head self-attention calculation using a cascaded SwinTransformer to obtain a second feature map, and perform cross-modal feature fusion on the first feature map and the second feature map to obtain a third feature map; The fifth module is used to evaluate the segmentation effect of the third feature map using cross entropy loss and Dice loss to obtain the tumor image segmentation model.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the tumor image segmentation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the tumor image segmentation method according to any one of claims 1 to 7.

Images