Tumor segmentation model training method and segmentation method based on two-level feature fusion

Abstract

The application provides a tumor segmentation model training method and segmentation method based on two-level feature fusion, relates to the field of medical image segmentation, and solves the technical problems that the boundary between the tumor and the surrounding tissue is unclear in the prior art, especially the low segmentation accuracy of the target region with complex shape and variable structure, and the easy occurrence of misjudgment or missed detection. The method comprises the following steps: obtaining a training sample set and a test sample set; constructing an image segmentation convolutional network; training the segmentation convolutional network based on the training sample set, cross-entropy loss and Dice loss; testing the tumor segmentation model based on the test sample set, and obtaining the tumor segmentation model. The application is used in the process of segmenting the tumor region in an ultrasound image and a magnetic resonance image.

Description

Technical Field

[0001] This application relates to the field of medical image segmentation, and in particular to a tumor segmentation model training method and segmentation method based on two-level feature fusion. Background Technology

[0002] With the rapid development and widespread application of medical imaging technology, imaging techniques such as MRI and ultrasound have become core tools in clinical diagnosis, surgical planning, and treatment outcome evaluation. Medical image segmentation, as a key step in image processing, aims to accurately separate specific structures (such as organs or lesion regions) in an image and has significant clinical application value.

[0003] In recent years, the rapid development of deep learning technology has greatly promoted the automation of medical image segmentation. Segmentation models based on CNN and Transformer have been widely used in various medical scenarios, significantly improving segmentation efficiency and accuracy, far exceeding traditional manual segmentation methods. However, when the boundary between tumors and surrounding tissues is unclear, the segmentation accuracy is low, and misjudgments or missed detections are prone to occur. Furthermore, for target regions with complex morphology and varied structures, such as breast tumors, existing models often struggle to obtain fine-grained segmentation results, failing to meet the needs of accurate clinical diagnosis. Summary of the Invention

[0004] This application provides a tumor segmentation method and apparatus based on two-level feature fusion, which solves the technical problem of unclear boundaries between tumors and surrounding tissues in the prior art, especially the low segmentation accuracy of target areas with complex morphology and varied structure, which is prone to misjudgment or missed detection.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, a training method for a tumor segmentation model based on two-level feature fusion is provided, including:

[0007] Obtain a training sample set and a test sample set; wherein, both the training sample set and the test sample set include multiple sets of sample images after preprocessing tumor images and several corresponding label images; the tumor images are ultrasound images or magnetic resonance images of tumors; the label images are images that have been segmented from the sample images and contain region labels;

[0008] Construct a segmentation convolutional network; wherein the segmentation convolutional network includes an embedding module, a transformer module, a decoder module, and a segmentation head module; the embedding module includes a ResNet backbone network and a CCDA module; wherein the CCDA module is a positional channel attention mechanism; the CCDA module consists of a positional attention mechanism layer and a channel attention mechanism layer.

[0009] The segmentation convolutional network is trained using a training sample set, cross-entropy loss, and Dice loss, and then tested using a test sample set to obtain a tumor segmentation model.

[0010] It should be noted that the labeled image is an image of the tumor region marked by domain experts. The tumor region includes the whole tumor, the tumor core, and the enhanced tumor.

[0011] Based on the above technical solution, in the tumor segmentation method based on two-level feature fusion provided in this application, the following is achieved:

[0012] In conjunction with the first aspect mentioned above, in one possible implementation, the ResNet backbone network consists of several initial convolutional layers and three residual layers. The initial convolutional layers have a 7×7 kernel, a stride of 2, and 64 output channels, while the residual layers have a 3×3 kernel.

[0013] In conjunction with the first aspect above, in one possible implementation, the tumor image is preprocessed, including:

[0014] The tumor image is rotated several times at arbitrary angles within a preset angle range to obtain several rotated images; the rotated images are then scaled to a preset input size to obtain several sample images.

[0015] In conjunction with the first aspect above, in one possible implementation, training the segmentation convolutional network includes:

[0016] Several sample images are input into the embedding module for feature processing to obtain the first feature;

[0017] The first feature is input to the transformer module for feature processing to obtain the second feature; wherein, the transformer module includes a hidden feature layer, layer normalization, a multi-head self-attention mechanism and a multilayer perceptron;

[0018] The second feature is input into the decoder module for feature processing to obtain the third feature; wherein, the decoder module includes a first upsampling layer, a gated fusion module, and a dual-branch attention fusion module;

[0019] The third feature is input into the segmentation head module for image segmentation to obtain a segmented image; wherein, the segmentation head module includes a 3×3 convolutional layer, a second upsampling layer and an output layer;

[0020] The segmentation convolutional network is updated based on the segmented image, label image, cross-entropy loss, and Dice loss, and an optimizer is used to obtain the updated segmentation convolutional network.

[0021] In conjunction with the first aspect above, in one possible implementation, the plurality of sample images are input to the embedding module for feature processing, including:

[0022] The preprocessed sample images from the training set are input into the ResNet backbone network for feature extraction;

[0023] The features output from the ResNet backbone are input into the CCDA module for feature enhancement to obtain the first feature.

[0024] In conjunction with the first aspect described above, in one possible implementation, the first feature is input to the converter module for feature processing, including:

[0025] The first feature is input into the transformer module and passes through the hidden feature layer, layer normalization, multi-head self-attention mechanism and multilayer perceptron in sequence to obtain the second feature.

[0026] In conjunction with the first aspect above, in one possible implementation, the second feature is input to the decoder module for feature processing, including:

[0027] After the second feature is input into the first upsampling layer for processing, it is superimposed through the dynamic serpentine convolutional layer in the gated fusion module and processed through the deformable convolutional layer in the gated fusion module to obtain superimposed information C and deformable convolutional information D.

[0028] The deformable convolutional information D and the superimposed information C are processed sequentially by average pooling, convolution, corrected linearization, convolution, and logistic S-function to obtain the channel information A.

[0029] After concatenating deformable convolutional information D and superimposed information C, a 7×7 convolution operation and logical S-function processing are performed to obtain spatial information B.

[0030] The channel information A, spatial information B, and superimposed information C are multiplied to obtain feature information E; the third feature is obtained by calculating Z=E×M+D×(1-M); where M is the dynamic fusion weight obtained by C and D after 1×1 convolution and the logistic S-function.

[0031] In conjunction with the first aspect described above, in one possible implementation, the third feature is input into the segmentation head module for image segmentation, including the following steps:

[0032] The third feature is input into a 3×3 convolutional layer to map the number of channels to the target category, and after processing by a second upsampling layer, it is output through the output layer to obtain the segmented image.

[0033] Secondly, a tumor segmentation method based on two-level feature fusion is provided, including:

[0034] Acquire the image to be segmented; the image to be segmented includes ultrasound image and magnetic resonance image;

[0035] The image to be segmented is input into the tumor segmentation model to obtain the segmented image corresponding to the ultrasound image and / or magnetic resonance image, wherein the tumor segmentation model is a model trained based on the model training method described in the first aspect above.

[0036] Thirdly, a processing device is provided, comprising: a communication unit and a processing unit; the communication unit is used to acquire a training sample set and a test sample set; the processing unit is used to construct a segmentation convolutional network; and to train the segmentation convolutional network based on the training sample set, cross-entropy loss and Dice loss, and then test the trained segmentation convolutional network based on the test sample set to obtain a tumor segmentation model.

[0037] Fourthly, this application provides a processing apparatus, comprising: a processor and a storage medium; the storage medium includes instructions, and the processor is configured to execute the instructions to implement the methods described in the first aspect and any possible implementation thereof. The processing apparatus may be an electronic device or a chip within an electronic device.

[0038] Fifthly, this application provides a tumor segmentation model training system based on two-level feature fusion, comprising: an information acquisition module and a model training module; wherein, the information acquisition module is used to acquire a training sample set and a test sample set; the model training module is used to construct a segmentation convolutional network; and, the segmentation convolutional network is trained based on the training sample set, cross-entropy loss and Dice loss, and then the trained segmentation convolutional network is tested based on the test sample set to obtain a tumor segmentation model.

[0039] Sixthly, this application provides a computer-readable storage medium storing instructions that, when executed on a processing device, cause the processing device to perform the methods described in the first aspect and any possible implementation thereof.

[0040] In a seventh aspect, this application provides a computer program product containing instructions that, when run on a processing device, cause the processing device to perform the methods described in the first aspect and any possible implementation thereof.

[0041] This application provides a tumor segmentation model training method and segmentation method based on two-level feature fusion. By constructing a novel segmentation convolutional network, which combines the advantages of the Transformer architecture and the local feature extraction capability of convolutional neural networks, it not only significantly improves the accuracy and reliability of image segmentation by introducing deformable convolution, dynamic serpentine convolution, and gated fusion mechanism, but also corrects and complements the relatively raw information brought by skip connections with upsampled information through positional channel attention and dual-branch attention fusion modules, thereby improving the segmentation accuracy and helping to realize tumor diagnosis and subsequent treatment.

[0042] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0043] Figure 1 A flowchart illustrating the training method for a tumor segmentation model based on two-level feature fusion provided in this application embodiment;

[0044] Figure 2 A schematic diagram illustrating the training process of the segmentation convolutional network provided in the embodiments of this application;

[0045] Figure 3 This is a structural diagram of the gating fusion module provided in the embodiments of this application;

[0046] Figure 4 This is a structural diagram of the dual-branch attention fusion module provided in an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of the structure of a processing device provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the hardware structure of a processing device provided in an embodiment of this application;

[0049] Figure 7 The structural diagram of the image segmentation method based on two-level feature fusion provided in the embodiments of this application is shown.

[0050] Figure 8 A structural diagram of the location channel attention mechanism layer provided in the embodiments of this application. Detailed Implementation

[0051] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0052] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0053] like Figure 1 As shown in the embodiments of this application, the tumor segmentation model training method based on two-level feature fusion includes:

[0054] S201. Obtain the training sample set and the test sample set.

[0055] The training sample set and the test sample set each include multiple sets of sample images after preprocessing tumor images and several corresponding label images; the tumor images are ultrasound images of tumors; the label images are images of sample images after segmentation and containing region labels.

[0056] In some implementations, the training and testing sample sets may include the BUSI breast tumor dataset and the brain tumor dataset.

[0057] The BUSI breast tumor dataset consists of 780 ultrasound images and corresponding segmentation maps of breast cancer from 600 female patients. Among them, there are 210 malignant cases, 437 benign cases, and 133 normal cases. In this embodiment, benign and malignant images (647 images in total) are used to validate the effectiveness of the proposed tumor segmentation method. All samples are randomly divided into a training set and a test set in an 8:2 ratio.

[0058] The brain tumor dataset was obtained from BraTS2020, an important public dataset for multimodal brain tumor segmentation widely used in the MICCAI Brain Tumor Segmentation Challenge. BraTS2020 contains 369 brain tumor patients, each with MRI scans in four different modalities (Flair, T1, T1ce, and T2), and tumor regions were labeled by domain experts. Labels included four classes: background, NCR / NET, ED, and ET. Evaluation was based on three different brain tumor regions: whole tumor (WT = NCR / NET + ED + ET), tumor core (TC = NCR / NET + ET), and enhanced tumor (ET). During preprocessing, each scan was 240×240×155 pixels. This embodiment uses 2D tumor image slices from the BraTS dataset as samples for training and testing. Based on this, scans of all modalities were sliced, with each slice being 224×224 pixels. This embodiment only uses the T1ce modality to validate the effectiveness of the proposed tumor segmentation method.

[0059] It should be noted that: Preprocessing of tumor images includes:

[0060] The tumor image is rotated several times at arbitrary angles within a preset angle range to obtain several rotated images. The rotated images are then scaled to a preset input size and preprocessed using a series of data augmentation techniques to improve the robustness and generalization ability of the model.

[0061] S202, Construct a segmented convolutional network;

[0062] The segmentation convolutional network consists of an embedding module, a transformer module, a decoder module, and a segmentation head module, arranged sequentially along the data transmission direction.

[0063] In some implementations, the embedded module includes the ResNet backbone network and the CCDA module; the CCDA module consists of a position attention mechanism layer and a channel attention mechanism layer.

[0064] The transformer module includes a hidden feature layer, layer normalization, a multi-head self-attention mechanism, and a multilayer perceptron;

[0065] The decoder module includes a first upsampling layer, a gated fusion module, and a dual-branch attention fusion module; the segmentation head module includes a 3×3 convolutional layer, a second upsampling layer, and an output layer.

[0066] It should be noted that the ResNet backbone consists of 50 layers of convolutional neural networks, including an initial convolutional layer and three residual layers; the initial convolutional layer has a 7×7 kernel, and the residual layer has a 3×3 kernel.

[0067] For example, after convolution through an initial 7×7 convolutional layer, the data is then processed by group normalization and ReLU activation before being fed into a 3×3 max pooling layer with a stride of 2 to further compress the spatial resolution. These operations transform the original 224×224×3 dimensionality into 56×56×64. Following this are three residual block stages: stage1, stage2, and stage3, containing 3, 4, and 9 residual blocks respectively. Each residual block consists of three convolutional layers: first, a 1×1 convolution for dimensionality reduction; then, a 3×3 convolution for feature extraction; and finally, a 1×1 convolution for dimensionality increase. Each convolutional layer is followed by group normalization and ReLU activation. The dimensions of stages1, 2, and 3 are 56×56×256, 28×28×512, and 14×14×1024, respectively. In the ResNet backbone, after the image is processed through these layers, the spatial size of the feature map gradually decreases (e.g., from 224x224 to 14x14), while the number of channels gradually increases, eventually outputting a high-dimensional feature representation with 2048 channels. After dimensionality reduction through a 1×1 convolution kernel, the final feature can be input into the CCDA module.

[0068] S203. The segmentation convolutional network is trained based on the training sample set, cross-entropy loss, and Dice loss, and then tested based on the test sample set to obtain the tumor segmentation model.

[0069] In one possible implementation of the embodiments of this application, combined with Figure 1 ,like Figure 2 As shown, the above S201 can be specifically implemented through the following S301, S302, S303, S304 and S305, which are explained in detail below:

[0070] The methods for training the segmentation convolutional network include:

[0071] S301: Input several sample images into the embedding module for feature processing to obtain the first feature;

[0072] S302: The first feature is input to the transformer module for feature processing to obtain the second feature; wherein, the transformer module includes a hidden feature layer, layer normalization, multi-head self-attention mechanism and multilayer perceptron;

[0073] S303: The second feature is input to the decoder module for feature processing to obtain the third feature; wherein, the decoder module includes a first upsampling layer, a gated fusion module and a dual-branch attention fusion module;

[0074] S304: Input the third feature into the segmentation head module to perform image segmentation and obtain a segmented image;

[0075] S305: Based on the segmented image, label map, cross-entropy loss, and Dice loss, the segmentation convolutional network is updated using an optimizer to obtain the updated segmentation convolutional network.

[0076] In some implementations, several sample images are input into the embedding module for feature processing, including the following steps:

[0077] The preprocessed sample images in the training set are input into the ResNet backbone network for feature extraction.

[0078] The features output from the ResNet backbone are input into the location attention mechanism layer for feature enhancement, resulting in the first feature.

[0079] For example, after the features are fed into the CCDA module, the number of input channels is first reduced from 768 to 48 through a 3x3 convolutional layer. Then, they are fed into the positional attention mechanism layer and the channel attention mechanism layer for calculation. The input information of the two attention mechanism layers is added together and then the number of channels is adjusted to 768 through a 1x1 convolution. The CCDA module can enhance the spatial and channel information of the features, providing more expressive input for subsequent modules.

[0080] In some implementations, the first feature is input to the transformer module for feature processing, including:

[0081] The first feature is input into the transformer module and passes through the hidden feature layer, multi-head self-attention mechanism and multilayer perceptron in sequence to reduce the dimension of the first feature and obtain the second feature.

[0082] For example, the first feature is input into the Transformer layer, thereby changing the dimension of the first feature from 14×14×768 to 196×768. The first feature with the 196×768 dimension is reshaped so that the first feature with the 196×768 dimension becomes 14×14×768. Then, a 1×1 convolution is applied to reduce the dimension, resulting in a second feature with the dimension of 14×14×512.

[0083] Please see Figure 3 The structural diagram of the gated fusion module shown, in some implementations, inputs the second feature to the decoder module for feature processing, including:

[0084] After the second feature is input into the first upsampling layer for processing, it is superimposed through the dynamic serpentine convolutional layer in the gated fusion module and processed through the deformable convolutional layer in the gated fusion module to obtain superimposed information C and deformable convolutional information D.

[0085] The deformable convolutional information D and the superimposed information C are processed sequentially by average pooling, convolution, corrected linearization, convolution, and logistic S-function to obtain the channel information A.

[0086] After concatenating deformable convolutional information D and superimposed information C, a 7×7 convolution operation and a logical S-function are performed to obtain spatial information B; after concatenating deformable convolutional information D and superimposed information C, a 1×1 convolution operation and a logical S-function are performed to obtain dynamic weight information F.

[0087] The channel information A, spatial information B, and superimposed information C are multiplied to obtain feature information E. The third feature is obtained by calculating Z=E×M+D×(1-M). Here, M is the dynamic fusion weight obtained by C and D after passing through a 1×1 convolution and a logistic S-function. The purpose of the D×(1-M) operation is to adjust the contribution ratio of deformable convolution in the fusion result according to the dynamic fusion weight.

[0088] It should be noted that the gated fusion module first introduces a 3×3 dynamic serpentine convolutional layer and two deformable convolutional layers. The sizes of these two deformable convolutional kernels are 7 and 9, respectively. After setting the x-axis and y-axis of the second feature, 7×7 or 9×9 convolutional kernels are used along the x and y axes. Based on this, the gated fusion module adaptively adjusts the weights of the dynamic serpentine convolutional layer and the deformable convolutional layer.

[0089] For example, please refer to Figure 3 As shown, the second feature is first input to an upsampling layer to improve resolution. Then, it passes through a dynamic serpentine convolutional layer with parallel 7x7 and 9x9 kernels. The learned x-axis and y-axis information are then superimposed to obtain superimposed information C. The second feature and the superimposed information C are then concatenated to obtain deformable convolutional information D. Finally, the superimposed information C and the deformable convolutional information D are concatenated. (See [link to documentation]). Figure 4 The diagram of the dual-branch attention fusion module structure is shown below. Then, the concatenated features are sequentially subjected to average pooling, 1×1 convolution, ReLU, 1×1 convolution and Sigmoid function (logistic sigmoid function). These operations are to obtain the channel information A of the features. At the same time, there are two branches that form three parallel branches with the above branches: the deformable convolution information D is simultaneously subjected to 7×7 convolution and Sigmoid function operation to obtain spatial information B; the obtained channel information A and spatial information B are multiplied with the information C from the previous stage to obtain information E; finally, E×M+D×(1-M) is used to obtain the final third feature Z. Figure 3In this context, G refers to the element-wise multiplication of the feature map after channel attention and spatial attention processing with the dynamic fusion weights. M and 1-M represent the contribution weights of the two input feature maps in the fusion process. The purpose of the H operation is to adjust the contribution ratio of deformable convolution in the fusion result according to the dynamic fusion weights.

[0090] It should be noted that the specific operation of the dual-branch attention fusion module is as follows: First, the output of the first upsampling layer and the skip connection features are concatenated along the channel dimension to obtain 1024×28×28 concatenated information. Among them, the skip connection features are the features passed through the ResNet backbone network output and the channel position attention mechanism layer. At the same time, there are two parallel branches: the average pooling branch and the max pooling branch. In these two branches, the average pooling layer, 1×1 convolution, and sigmoid function are passed in sequence, and the max pooling layer, 1×1 convolution and sigmoid function are passed in sequence. Then, the information obtained from the two pooling branches is concatenated again. Then, the weight information obtained by dimensionality reduction through 1×1 convolution is multiplied by the information obtained by concatenating the upsampling features and skip connection features along the channel dimension and then dimensionality reduction is performed to obtain a feature dimension of 512×28×28. The operation at this point is to obtain the corresponding channel information. The next step is to obtain the corresponding spatial information: the output of the first upsampling layer and the skip connection features are compressed by performing a 1×1 convolution to extract as much spatial information as possible. Then, the two sets of compressed spatial information are superimposed and passed through the Sigmoid function, and then multiplied by the information finally obtained from the double pooling branch. At this point, the dimension size is still 512×28×28. The Sigmoid function restricts the output value of the channel attention to between 0 and 1, so that it can be used as attention weights.

[0091] In some implementations, the third feature is input into the segmentation head module for image segmentation, including the following steps:

[0092] The third feature is input into a 3×3 convolutional layer to map the number of channels to the target category, and the resolution is increased to 224×224 through a second upsampling layer. Finally, a 224×224 segmented image is output through the output layer.

[0093] Furthermore, in this embodiment, the proposed network is implemented in the PyTorch framework, using a stochastic gradient descent (SGD) optimizer with a momentum of 0.9 and a weight decay of 1×1. The initial learning rate was . The training process was completed on one A40.

[0094] Table 1 shows a comparison of the segmentation performance of the proposed method in this embodiment with that of three traditional segmentation models (Unet, TransUne, and DA-TransUnet) on the BUSI dataset:

[0095] Table 1

[0096]

[0097] Table 2 shows a comparison of the segmentation performance of the proposed method in this embodiment with three traditional segmentation models (Unet, TransUne, and DA-TransUnet) on the BUSI dataset:

[0098] Table 2

[0099]

[0100] The foregoing mainly describes the solutions of the embodiments of this application from the perspective of device implementation. It is understood that each device, such as a processing apparatus, includes at least one of the hardware structures and software modules corresponding to the execution of each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] This embodiment also provides a tumor segmentation method based on two-level feature fusion, including:

[0102] An ultrasound image is acquired and input into a tumor segmentation model to obtain a segmented image of the image to be segmented, wherein the tumor segmentation model is a model trained based on the model training method described in the first aspect above.

[0103] This application embodiment can divide the processing device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0104] When using integrated units, Figure 5A possible structural schematic diagram of the processing device (referred to as processing device 50) involved in the above embodiments is shown. The processing device 50 includes a processing unit 501 and a communication unit 502, and may also include a storage unit 503. Figure 5 The schematic diagram shown can be used to illustrate the structure of the processing device involved in the above embodiments.

[0105] when Figure 5 The schematic diagram shown is used to illustrate the structure of the processing device involved in the above embodiments. The processing unit 501 is used to control and manage the operation of the processing device, the communication unit 502 is used for the processing device to communicate with other devices, and the storage unit 503 is used to store the program code and data of the processing device.

[0106] For example, communication unit 502 is used to acquire training sample sets and test sample sets.

[0107] Processing unit 502 is used to construct a segmentation convolutional network; and to train the segmentation convolutional network based on a training sample set, cross-entropy loss and Dice loss, and then test the trained segmentation convolutional network based on a test sample set to obtain a tumor segmentation model.

[0108] The processing unit 501 can be a processor or a controller, and the communication unit 502 can be a communication interface, transceiver, transceiver circuit, transceiver device, etc. The term "communication interface" is a general term and may include one or more interfaces. The storage unit 503 can be a memory. When the processing device 50 is a chip, the processing unit 501 can be a processor or a controller, and the communication unit 502 can be an input interface and / or an output interface, pins, or circuits, etc. The storage unit 503 can be a storage unit within the chip (e.g., a register, cache, etc.) or a storage unit located outside the chip (e.g., read-only memory (ROM), random access memory (RAM, etc.).

[0109] The communication unit can also be called a transceiver unit. The antenna and control circuit with transceiver functions in the processing device 50 can be considered as the communication unit 502 of the processing device 50, and the processor with processing functions can be considered as the processing unit 501 of the processing device 50. Optionally, the device in the communication unit 502 that implements the receiving function can be considered as a communication unit, which is used to execute the receiving steps in the embodiments of this application. The communication unit can be a receiver, a receiver circuit, etc. The device in the communication unit 502 that implements the transmitting function can be considered as a transmitting unit, which is used to execute the transmitting steps in the embodiments of this application. The transmitting unit can be a transmitter, a transmitter, a transmitting circuit, etc.

[0110] Figure 5 If the integrated units in the process are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. Storage media for storing computer software products include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0111] Figure 5 The units in the process can also be called modules; for example, a processing unit can be called a processing module.

[0112] This application also provides a hardware structure diagram of a processing device (referred to as processing device 60), see [link to diagram]. Figure 6 The processing device 60 includes a processor 601, and optionally, a memory 602 connected to the processor 601.

[0113] In the first possible implementation, see Figure 6 The processing device 60 also includes a transceiver 603. The processor 601, memory 602, and transceiver 603 are connected via a bus. The transceiver 603 is used to communicate with other devices or communication networks. Optionally, the transceiver 603 may include a transmitter and a receiver. The device in the transceiver 603 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 603 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0114] Based on the first possible implementation method Figure 6 The schematic diagram shown can be used to illustrate the structure of the processing device involved in the above embodiments.

[0115] in, Figure 6 Alternatively, the system chip in the processing device can be illustrated. In this case, the actions performed by the aforementioned processing device can be implemented by the system chip, and the specific actions performed can be found above, and will not be repeated here.

[0116] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0117] The processor in this application may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., which are various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor may be a separate semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it may be integrated with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits) to form a SoC (System-on-a-Chip), or it may be integrated as a built-in processor within an ASIC. The ASIC with the integrated processor may be packaged separately or together with other circuits. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0118] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; or electrically erasable programmable-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0119] This application also provides a computer-readable storage medium including instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0120] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described above.

[0121] This application also provides a chip including a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is used to run computer programs or instructions to implement the above-described method. The interface circuit is used to communicate with other modules outside the chip.

[0122] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0123] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0124] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method for training a tumor segmentation model based on two-level feature fusion, characterized in that, The method comprises the following steps: obtaining a training sample set and a test sample set; wherein the training sample set and the test sample set each comprise a plurality of groups of sample images after preprocessing of tumor images and a plurality of corresponding label images; the label image is an image after segmentation of the sample image; constructing a segmentation convolutional network; wherein the segmentation convolutional network comprises an embedding module, a transformer module, a decoder module and a segmentation head module; the embedding module comprises a ResNet backbone network and a CCDA module; the CCDA module is composed of a position attention mechanism layer and a channel attention mechanism layer; training the segmentation convolutional network based on the training sample set, cross-entropy loss and Dice loss, testing the trained segmentation convolutional network based on the test sample set, and obtaining a tumor segmentation model; training the segmentation convolutional network comprises: inputting a plurality of sample images into the embedding module for feature processing to obtain first features; inputting the first features into the transformer module for feature processing to obtain second features; wherein the transformer module comprises a hidden feature layer, layer normalization, a multi-head self-attention mechanism and a multi-layer perceptron; inputting the second features into the decoder module for feature processing to obtain third features; wherein the decoder module comprises a first upsampling layer, a gated fusion module and a double-branch attention fusion module; inputting the third features into the segmentation head module for image segmentation to obtain a segmentation image; wherein the segmentation head module comprises a 3x3 convolutional layer, a second upsampling layer and an output layer; updating the segmentation convolutional network based on the segmentation image, the label image, the cross-entropy loss and the Dice loss through an optimizer to obtain an updated segmentation convolutional network; inputting the second features into the decoder module for feature processing comprises: after processing the second features in the first upsampling layer, the stacked information C and the deformable convolution information D are obtained by respectively stacking through the dynamic snake convolutional layer in the gated fusion module and processing through the deformable convolutional layer in the gated fusion module; the channel information A is obtained by sequentially performing average pooling, convolution, rectified linear, convolution and logistic S function processing on the deformable convolution information D and the stacked information C; the spatial information B is obtained by performing 7x7 convolution operation and logistic S function processing on the concatenated deformable convolution information D and stacked information C; the feature information E is obtained by point multiplication operation of the channel information A, the spatial information B and the stacked information C; the third features are obtained by calculating Z=E x M+D x (1-M); wherein M is the dynamic fusion weight obtained by 1x1 convolution and logistic S function of C and D. 2.The two-stage feature fusion based tumor segmentation model training method of claim 1, wherein, inputting the plurality of sample images into the embedding module for feature processing comprises: inputting the preprocessed sample images in the training set into the ResNet backbone network for feature extraction; inputting the features output by the ResNet backbone network into the CCDA module for feature enhancement to obtain the first features. 3.The method of claim 1, wherein, inputting the first features into the transformer module for feature processing comprises: The first feature is input into a transformer module and sequentially passes through a hidden feature layer, layer normalization, a multi-head self-attention mechanism and a multi-layer perceptron to obtain a second feature. 4.The method of claim 1, wherein, The third feature is input into a segmentation head module for image segmentation, including: The third feature is input into a 3x3 convolution layer to map the number of channels to the target class, and after processing by a second upsampling layer, the segmentation image is output by an output layer. 5.The method of claim 1, wherein, Based on the segmentation image, the label image, the cross-entropy loss and the Dice loss, the segmentation convolutional network is updated by an optimizer, including: The cross-entropy loss and the Dice loss are combined by weighting to obtain a loss function, the loss of the segmentation image and the label image is calculated by the loss function, and the model parameters in the segmentation convolutional network are optimized based on the optimizer until the loss is less than a loss threshold, and the model parameters in the segmentation convolutional network are saved. 6.The method of training a tumor segmentation model based on two-level feature fusion according to claim 1, wherein, The ResNet backbone network includes an initial convolution layer and three residual layers; the convolution kernel of the initial convolution layer is 7x7, and the convolution kernel of the residual layer is 3x3.

7. The two-stage feature fusion-based tumor segmentation model training method according to claim 1, characterized in that, The tumor image is preprocessed, including: The tumor image is rotated at any angle within a preset angle range for several times to obtain several rotated images; the rotated images are scaled to a preset input size to obtain several sample images.

8. The image segmentation method based on two-stage feature fusion, characterized in that, Including: An image to be segmented is obtained; The image to be segmented is input into the tumor segmentation model to obtain a segmentation image of the image to be segmented; wherein the tumor segmentation model is a model trained based on the tumor segmentation model training method based on two-level feature fusion according to any one of claims 1-7.

Images