A CT tumor image segmentation method and system based on a multi-attention U-shaped network

Abstract

The application discloses a CT tumor image segmentation method and system based on a U-shaped network with multiple attention mechanisms, and first, a data set containing original CT images and tumor segmentation results is established; second, a U-shaped network model with a multiple attention mechanism is constructed; then, a loss function of the U-shaped network model is constructed, next, the U-shaped network model is trained by using a training data set; finally, the trained model is integrated into software to realize visualization of tumor segmentation. The application is a method capable of automatically segmenting CT images, and the core is an improved U-shaped network structure model. The model increases an attention mechanism and a skip link for expanding a growth distance on the basis of UNet, so as to extract main information of an image and provide a larger receptive field, and the problem that UNet lacks operation between pixels far away from each other is solved. The improved model is embedded into a graphical user interface, and the problem of difficult reading is effectively solved.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and relates to a CT tumor image segmentation method and system based on a multi-attention U-shaped network. Background Technology

[0002] Existing liver tumor segmentation methods are mainly divided into two types: machine learning-based methods and deep learning-based methods. Traditional machine learning-based methods include thresholding, level sets, and clustering. These methods generally rely on doctors specifying regions of interest or seed points, making them semi-automatic and inefficient. CT images also suffer from blurred boundaries, leading to relatively low accuracy after processing with traditional methods. With the increasing computing power and the continuous development of deep learning technology, more and more researchers have begun to use deep learning for automatic liver tumor segmentation in recent years. However, convolution has the disadvantage of a small receptive field, making it difficult to establish long-distance pixel connections. Furthermore, tumors often have blurred boundaries in imaging, causing existing methods to easily exhibit undersegmentation or oversegmentation at tumor boundaries. Moreover, the models often lack graphical interfaces, making them difficult to apply. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies, such as the small receptive field of convolution, which makes it impossible to establish long-distance pixel connections, and the fact that tumors often have blurred boundaries in imaging, leading to undersegmentation or oversegmentation of tumor boundaries in existing methods. This invention provides a CT tumor image segmentation method and system based on a multi-attention U-shaped network.

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] This invention proposes a CT tumor image segmentation method based on a multi-attention U-shaped network, comprising the following steps:

[0006] A U-shaped network model with a multi-attention mechanism is constructed based on the UNet network;

[0007] Construct the loss function for the U-shaped network model based on the U-shaped network model;

[0008] The CT image is input into the U-shaped network model, and the segmented CT image is output to achieve tumor segmentation in the CT image.

[0009] Preferably, the U-shaped network model with multiple attention mechanisms includes a Unet Backbone module, a visual attention module, and a blending attention module.

[0010] Preferably, the Unet Backbone module is constructed as follows: a five-layer structure, the first four layers include two convolutional modules and one pooling module, and the last layer includes two convolutional layers; the output of the first layer is used as the input of the second layer and the input of visual attention 1, the output of the second layer is used as the input of the third layer and the input of visual attention 2, the output of the third layer is used as the input of the fourth layer and the input of visual attention 3, the output of the fourth layer is used as the input of the fifth layer and the input of visual attention 4, and the output of the fifth layer is used as the input of visual attention 5 and the output of the overall structure;

[0011] The first to fourth convolutional modules are each composed of two 3×3 convolutional layers and one 2×2 max pooling layer connected in sequence, while the fifth convolutional module is composed of two 3×3 convolutional layers connected in sequence.

[0012] Preferably, the visual attention module is constructed as follows: it consists of multiple branches, each branch is composed of an attention mechanism consisting of a non-dilated convolution A and a dilated convolution B. The output of convolution A is the input of convolution B. The input of convolution B is multiplied by the output of convolution A after a sigmoid operation, which is the output of that branch. The outputs of all branches are integrated by a convolution. The size of the convolution kernel used for integration is 1*1.

[0013] The output of the fifth layer serves as the input to blending attention 5 and the fourth layer; the output of the fourth layer serves as the input to blending attention 4 and the third layer; the output of the third layer serves as the input to blending attention 3 and the second layer; the output of the second layer serves as the input to blending attention 2 and the first layer; the output of the first layer serves as the input to blending attention 1 and the overall output.

[0014] The number of branches varies across layers: the first layer has five branches, the second has four, the third has three, the fourth has two, and the fifth has one. The kernel size varies across branches, and the relationship between the branch depth (depth1) and the kernel size (kernel_size1) of the convolution A follows this formula:

[0015] kernel_size1 = 2 * depth1 + 1

[0016]

[0017] The relationship between the branch depth2 and the convolution kernel_size2 satisfies the following formula:

[0018]

[0019] Preferably, the blending attention module is constructed as follows: the output of the fifth VA module is subjected to 16-fold bilinear interpolation, the output of the fourth VA module is subjected to 8-fold bilinear interpolation, the output of the third VA module is subjected to 4-fold bilinear interpolation, the output of the second VA module is subjected to 2-fold bilinear interpolation, and then concatenated with the output of the first VA module. The fuse is obtained by processing with three convolutions.

[0020] For the fifth layer BA, the fuse is downsampled by 16 times. Then, the weights w1 and w2 are calculated for the input of the fifth layer and the downsampled fuse, respectively. After processing with three convolutions, the weights are input into the sigmoid function for self-attention processing. The function is multiplied by itself, and the two inputs are concatenated and then processed with a convolution.

[0021] For the fourth layer BA, the fuse is downsampled by 8 times. Then, the weights w1 and w2 are calculated for the input of the fourth layer VA and the downsampled fuse, respectively. After processing with three convolutions, they are input into the sigmoid function for self-attention processing. The function is multiplied by itself, and the two inputs are concatenated and processed with a convolution.

[0022] For the third layer BA, the fuse is downsampled by 4 times. Then, the weights w1 and w2 are calculated for the input of the third layer VA and the downsampled fuse, respectively. After processing with three convolutions, the weights are input into the sigmoid function for self-attention processing. The function is multiplied by itself, and the two inputs are concatenated and then processed with a convolution.

[0023] For the second layer BA, the fuse is downsampled by a factor of 2. Then, the weights w1 and w2 are calculated for the input of the second layer VA and the downsampled fuse, respectively. After processing with three convolutions, the weights are input into the sigmoid function for self-attention processing, multiplied by itself, and the two inputs are concatenated and processed with a single convolution.

[0024] For the first layer BA, the weights w1 and w2 are calculated for the inputs of fuse and the fifth layer VA, respectively. After processing them with three convolutions, they are input into the sigmoid function for self-attention processing. The inputs are multiplied by the function itself, and the two inputs are concatenated and then processed with a single convolution.

[0025] The outputs of the five-layer BA module are upsampled by 16, 8, 4, 2, and 1 times respectively, then concatenated, processed by a convolution, and used as the output.

[0026] The three convolutional kernels used for splicing are 1*1, 3*3, and 3*3 in size; the convolutional kernels in the self-attention layer of the BA layer are 1*1, 3*3, and 3*3 in size; the convolutional kernel for splicing is 1*1 in size; and the convolutional kernel for final processing is 1*1 in size.

[0027] Preferably, the process of constructing the loss function of the DAU-Net network model is as follows:

[0028] First, the DICE coefficient is used to construct the first loss function L1 for downsampling, as shown in the following formula:

[0029]

[0030] Where X1 represents the output image of the Unet Backbone module;

[0031] Second, the DICE coefficient is used to construct the second loss function L2 for downsampling, as shown in the following formula:

[0032]

[0033] Where X2 represents the output image of the visual attention module;

[0034] Third, the DICE coefficients are used to construct the third loss function L3 for downsampling, as shown in the following formula:

[0035]

[0036] Where Y represents the gold standard for segmentation, and X3 represents the output image of the blending attention module;

[0037] Fourth, combine the first loss function L1, the second loss function L2, and the third loss function L3 to construct the loss function L=(L1+L2)*α+L3 of the DAU-Net network.

[0038] Preferably, PySimpleGUI is used to visualize tumor segmentation.

[0039] This invention proposes a system for CT tumor image segmentation based on a multi-attention U-shaped network, comprising:

[0040] A model building module, which is used to build a U-shaped network model with a multi-attention mechanism based on the UNet network;

[0041] A loss function construction module, which is used to construct a loss function for the U-shaped network model based on the U-shaped network model;

[0042] The image segmentation module is used to input CT images into a U-shaped network model and output segmented CT images to achieve tumor segmentation in CT images.

[0043] The present invention proposes a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a CT tumor image segmentation method based on a multi-attention U-shaped network.

[0044] The present invention proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a CT tumor image segmentation method based on a multi-attention U-shaped network.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] This invention proposes a CT tumor image segmentation method based on a multi-attention U-shaped network. Compared with existing technologies, it addresses the lack of long-range dependencies in existing networks by using two different attention mechanisms concatenated to extract global capabilities. This invention provides an automatic CT image segmentation method, with an improved U-shaped network structure model, DAU-Net, which can improve the low accuracy of current tumor segmentation methods. Based on UNet, this model adds an attention mechanism and expands long-range skip links to extract key image information and provide a larger receptive field, solving the problem of insufficient computation between distant pixels in UNet.

[0047] Furthermore, embedding the model into the user's graphical interface facilitates operation and makes the results more intuitive.

[0048] This invention proposes a CT tumor image segmentation system based on a multi-attention U-shaped network. By dividing the system into a model building module, a loss function construction module, and an image segmentation module, tumor image segmentation is achieved. The modular approach ensures that each module is independent, facilitating unified management of all modules. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the CT tumor image segmentation method based on a multi-attention U-shaped network according to the present invention.

[0051] Figure 2 This is a schematic diagram of the liver segmentation structure of the present invention.

[0052] Figure 3 This is a schematic diagram of the VA module structure of the present invention.

[0053] Figure 4 This is a schematic diagram of the BA module structure of the present invention.

[0054] Figure 5 This is a schematic diagram of the user graphical interface of the present invention.

[0055] Figure 6 This is a diagram of the CT tumor image segmentation system based on a multi-attention U-shaped network according to the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0058] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0059] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0061] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0062] The present invention will now be described in further detail with reference to the accompanying drawings:

[0063] This invention proposes a CT tumor image segmentation method based on a multi-attention U-shaped network, such as... Figure 1 As shown, it includes the following steps:

[0064] S1. Construct a U-shaped network model with a multi-attention mechanism based on the UNet network;

[0065] S2. Construct the loss function for the U-shaped network model based on the U-shaped network model;

[0066] S3. Input the CT image into the U-shaped network model and output the segmented CT image to realize tumor segmentation in the CT image.

[0067] like Figure 2 As shown, the specific steps include the following:

[0068] Step 1: Obtain the original training dataset containing the original CT images and the patient's tumor stage, and manually segment the liver and tumor based on the patient's CT findings. The specific process includes:

[0069] 1.1 For the patient's CT images, a window with a width of 400 and a window level of 0 is applied. The following formula is used to process pixel P:

[0070]

[0071] 1.2. For the CT image I after windowing, anisotropic filtering is performed. The principle is as follows:

[0072]

[0073]

[0074] Where div is the divergence operator, I is the input CT image, and c(·) is the diffusion coefficient.

[0075] 1.3. For image I after anisotropic filtering, seed growth is performed for dataset labeling. The steps are as follows:

[0076] For image I, select n points as initial seeds to form a grown region. Determine whether the neighborhood formed by the surrounding s points satisfies rule f. If it does, include it in the grown region; otherwise, end the operation. Continue until all neighborhoods no longer satisfy the growth rule.

[0077] 1.4 For image I after the region growth is completed, dilation and erosion are performed to fill the holes. The dilation formula is as follows.

[0078]

[0079]

[0080] In the formula: A is the image to be processed; B is a user-defined structuring element. The above formula represents eroding image A using structure B and dilating image A using structure B.

[0081] 1.5. Manually calibrate the images after all processing to obtain the dataset.

[0082] Step 2: Construct a U-shaped network model with a multi-attention mechanism, called DAU-Net. This network model includes Unet Backbone, visual attention, and blending attention modules. The specific construction process includes:

[0083] 2.1 Construct the Unet Backbone module, which is used solely for downsampling. It consists of five layers. The first four layers include two convolutional modules and one pooling module to achieve downsampling. The last layer includes two convolutional layers. The output of the first layer serves as the input to the second layer and visual attention 1; the output of the second layer serves as the input to the third layer and visual attention 2; the output of the third layer serves as the input to the fourth layer and visual attention 3; the output of the fourth layer serves as the input to the fifth layer and visual attention 4; and the output of the fifth layer serves as the input to visual attention 5 and the overall structure's output.

[0084] 2.2 In the initial segmentation module described in step 2.1, the first to fourth convolutional modules are each composed of two 3×3 convolutional layers and one 2×2 max pooling layer connected in sequence, and the fifth convolutional module is composed of two 3×3 convolutional layers connected in sequence.

[0085] 2.3, such as Figure 3 As shown, a visual attention module is constructed. The VA (visual attention) module consists of multiple branches. Each branch is an attention mechanism composed of a non-dilated convolution A and a dilated convolution B. The output of convolution A is the input of convolution B. The input of convolution B is multiplied by the output of convolution A after a sigmoid operation, which is the output of that branch. The outputs of all branches are processed by a single convolution. There are five VA layers in total. The output of the fifth VA layer serves as the input to blending attention 5 and the fourth VA layer; the output of the fourth VA layer serves as the input to blending attention 4 and the third VA layer; the output of the third VA layer serves as the input to blending attention 3 and the second VA layer; the output of the second VA layer serves as the input to blending attention 2 and the first VA layer; and the output of the first VA layer serves as the input to blending attention 1 and the overall output.

[0086] 2.4 In the visual attention module described in step 2.3, the number of branches in each VA module is inconsistent. The first VA module has five branches, the second VA module has four branches, the third VA module has three branches, the fourth VA module has two branches, and the fifth VA module has one branch. The convolutional kernel size of each branch is inconsistent, and the relationship between the branch depth and the convolutional A size satisfies the following formula:

[0087] kernel_size = 2 * depth + 1

[0088]

[0089] The relationship between branch depth and convolution size B satisfies the following formula:

[0090] kernel_size = 2 * depth + 1

[0091] padding = kernel_size

[0092] The size of the convolutional kernel used for integration is 1*1;

[0093] 2.5, such as Figure 4As shown, a blending attention module is constructed. First, the output of the fifth layer VA module is subjected to 16x bilinear interpolation, the output of the fourth layer VA module is subjected to 8x bilinear interpolation, the output of the third layer VA module is subjected to 4x bilinear interpolation, and the output of the second layer VA module is subjected to 2x bilinear interpolation. These are concatenated with the output of the first layer VA module and processed using three convolutions to obtain a fuse. For the fifth layer BA, the fuse is downsampled by 16x. Then, weights w1 and w2 are calculated for the input of the fifth layer VA and the downsampled fuse, respectively. These are then processed using three convolutions and input into a sigmoid function for self-attention processing. The weights are multiplied by the sigmoid function, and the two resulting inputs are concatenated and processed using a single convolution. For the fourth layer BA, the fuse is downsampled by 8x. Then, weights w1 and w2 are calculated for the input of the fourth layer VA and the downsampled fuse, respectively. These are then processed using three convolutions and input into a sigmoid function for self-attention processing. The weights are multiplied by the sigmoid function, and the two resulting inputs are concatenated and processed using a single convolution. For the third layer BA, the fuse is downsampled by a factor of 4. Then, weights w1 and w2 are calculated for the input of the third layer VA and the downsampled fuse, respectively. These weights are then processed by three convolutions and fed into a sigmoid function for self-attention processing. The sigmoid function is multiplied by itself, and the two resulting inputs are concatenated and processed by a single convolution. For the second layer BA, the fuse is downsampled by a factor of 2. Then, weights w1 and w2 are calculated for the input of the second layer VA and the downsampled fuse, respectively. These weights are then processed by three convolutions and fed into a sigmoid function for self-attention processing. The sigmoid function is multiplied by itself, and the two resulting inputs are concatenated and processed by a single convolution. For the first layer BA, weights w1 and w2 are calculated for the input of the fuse and the fifth layer VA, respectively. These weights are then processed by three convolutions and fed into a sigmoid function for self-attention processing. The sigmoid function is multiplied by itself, and the two resulting inputs are concatenated and processed by a single convolution. The outputs of the five-layer BA module are upsampled by 16, 8, 4, 2, and 1 times respectively, then concatenated, processed by a convolution, and used as the output.

[0094] 2.6 In the visual attention module of step 2.5, the three convolutional kernels used for concatenation are 1*1, 3*3, and 3*3 in size, respectively. The convolutional kernels in the self-attention of the BA layer are 1*1, 3*3, and 3*3 in size, and the convolutional kernel used for concatenation is 1*1 in size. The final convolutional kernel used for processing is 1*1 in size.

[0095] 2.7. The outputs of 2.1, 2.3, and 2.5 are processed by a convolution and then concatenated. The final output is processed by a single convolution.

[0096] Step 3: Construct the loss function for the DAU-Net network model. The specific process includes:

[0097] 3.1 The DICE coefficient is used to construct the downsampling loss function L1, as shown in the following formula:

[0098]

[0099] Here, Y represents the gold standard for segmentation. For liver segmentation, the liver and liver tumor regions are represented by "1", and other regions are represented by "0". For tumor segmentation, the liver tumor region is represented by "1", and other regions are represented by "0". X1 represents the output image of the UnetBackbone module.

[0100] 3.2. The DICE coefficient is used to construct the downsampling loss function L2, as shown in the following formula:

[0101]

[0102] Here, Y represents the gold standard for segmentation. For liver segmentation, the liver and liver tumor regions are represented by "1", and other regions by "0". For tumor segmentation, the liver tumor region is represented by "1", and other regions by "0". X2 represents the output image of the visual attention module.

[0103] 3.3. The DICE coefficient is used to construct the downsampling loss function L3, as shown in the following formula:

[0104]

[0105] Here, Y represents the gold standard for segmentation. For liver segmentation, the liver and liver tumor regions are represented by "1", and other regions by "0". For tumor segmentation, the liver tumor region is represented by "1", and other regions by "0". X3 represents the output image of the blendingattention module.

[0106] 3.4 Constructing the loss function L of the DAU-Net network by combining L1, L2, and L3:

[0107] L=(L1+L2)*α+L3

[0108] Step 4: Use the training dataset to train the DAU-Net network for liver segmentation until L converges, and obtain model A.

[0109] Step 5: Multiply the extracted liver region ROI with the original image to obtain a training set containing only the liver portion. Use this training set to train DAU-Net for liver tumor segmentation until L converges, resulting in model B.

[0110] Step 6: Combine models A and B to obtain a two-segmented model.

[0111] like Figure 5 As shown, PySimpleGUI is used for graphical interface development, and a U-shaped network model is embedded in it.

[0112] This invention proposes a CT tumor image segmentation system based on a multi-attention U-shaped network, such as... Figure 6 As shown, it includes a model building module, a loss function building module, and an image segmentation module;

[0113] The model building module is used to build a U-shaped network model with a multi-attention mechanism based on the UNet network;

[0114] The loss function construction module is used to construct the loss function of the U-shaped network model based on the U-shaped network model;

[0115] The image segmentation module is used to input CT images into a U-shaped network model and output segmented CT images to achieve tumor segmentation in CT images.

[0116] An embodiment of the present invention provides a terminal device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the various method embodiments described above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.

[0117] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention.

[0118] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0119] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0120] The memory can be used to store the computer program and / or module. The processor implements various functions of the terminal device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory.

[0121] If the modules / units integrated into the terminal device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0122] This invention proposes a CT tumor image segmentation method based on a multi-attention U-shaped network, comprising the following steps: 1) establishing a dataset containing original CT images and liver segmentation and classification results; 2) constructing a U-shaped network model with a multi-attention mechanism; 3) constructing a network loss function; 4) training the network model using a training dataset; and 5) integrating the trained model into software to obtain the final tumor segmentation. The core of this invention is an improved U-shaped network structure model. Based on UNet, this model adds an attention mechanism and expands long-distance skip links to extract key image information and provide a larger receptive field, solving the problem of insufficient computation between distant pixels in UNet. The improved model, embedded in a graphical user interface, effectively solves the problem of difficult image interpretation. The DAU-Net model can improve the current problem of low accuracy in tumor recognition. Compared with existing technologies, the main advantages and innovations of the present invention include: 1) addressing the problem of lack of long-distance dependencies in networks by using two different attention mechanisms to extract global capabilities in series; 2) classifying the extracted tumor results based on prior knowledge on a self-established dataset; and 3) embedding the model into the user's graphical interface for easy operation and more intuitive results.

[0123] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A CT tumor image segmentation method based on a multi-attention U-shaped network, characterized in that, Includes the following steps: A U-shaped network model with a multi-attention mechanism is constructed based on the UNet network; Construct the loss function for the U-shaped network model based on the U-shaped network model; The CT image is input into the U-shaped network model, and the segmented CT image is output to achieve tumor segmentation in the CT image; The U-shaped network model with multiple attention mechanisms includes the Unet Backbone module, the visual attention module, and the blending attention module; The visual attention module is constructed as follows: it consists of multiple branches. Each branch is composed of an attention mechanism consisting of a non-dilated convolution A and a dilated convolution B. The output of convolution A is the input of convolution B. The input of convolution B is multiplied by the output of convolution A after a sigmoid operation, which is the output of that branch. The outputs of all branches are integrated by a convolution. The size of the convolution kernel used for integration is 1*1. In the visual attention module, the output of the fifth layer serves as the input to blending attention 5 and the fourth layer; the output of the fourth layer serves as the input to blending attention 4 and the third layer; the output of the third layer serves as the input to blending attention 3 and the second layer; the output of the second layer serves as the input to blending attention 2 and the first layer; and the output of the first layer serves as the input to blending attention 1 and the overall output. The blending attention module includes blending attention 1, blending attention 2, blending attention 3, blending attention 4, and blending attention 5. In the visual attention module, the number of branches varies across layers: the first layer has five branches, the second has four, the third has three, the fourth has two, and the fifth has one. The kernel size and branch depth also vary between layers. With convolution A The relationship between their sizes satisfies the following formula: branch depth With convolution B The relationship between their sizes satisfies the following formula: ； The blending attention module is constructed as follows: the output of the fifth VA module is subjected to 16x bilinear interpolation, the output of the fourth VA module is subjected to 8x bilinear interpolation, the output of the third VA module is subjected to 4x bilinear interpolation, and the output of the second VA module is subjected to 2x bilinear interpolation. These are then concatenated with the output of the first VA module and processed using three convolutions to obtain the fuse. Here, the VA module is the visual attention module. For the fifth layer BA, the fuse is downsampled by a factor of 16. Weights w1 and w2 are calculated for both the input to the fifth layer and the downsampled fuse, and then processed using three convolutions before being input into the sigmoid function for self-attention. The weights are multiplied by the sigmoid function, and the two resulting inputs are concatenated and processed using a single convolution. For the fourth layer BA, the fuse is downsampled by a factor of 8. Weights w1 and w2 are calculated for both the input to the fourth layer VA and the downsampled fuse, and then processed using three convolutions before being input into the sigmoid function for self-attention. The weights are multiplied by the sigmoid function, and the two resulting inputs are concatenated and processed using a single convolution. For the third layer BA, the fuse is downsampled by a factor of 4. Weights w1 and w2 are calculated for both the input to the third layer VA and the downsampled fuse, and then processed using three convolutions before being input into the sigmoid function. The sigmoid function performs self-attention processing, multiplying itself by the sigmoid function. The two inputs are then concatenated and processed using a convolution. For the second layer BA, the fuse is downsampled by a factor of 2. Weights w1 and w2 are calculated for the inputs of the second layer VA and the downsampled fuse, respectively. These weights are then processed using three convolutions before being input into the sigmoid function for self-attention processing. The sigmoid function is multiplied by the sigmoid function, and the two inputs are then concatenated and processed using a convolution. For the first layer BA, weights w1 and w2 are calculated for the inputs of the fuse and the fifth layer VA, respectively. These weights are then processed using three convolutions before being input into the sigmoid function for self-attention processing. The sigmoid function is multiplied by the sigmoid function, and the two inputs are then concatenated and processed using a convolution. The outputs of the fifth layer BA module are upsampled by factors of 16, 8, 4, 2, and 1, respectively, concatenated, and processed using a convolution before being output. Loss function of DAU-Net network Among them, the first loss function L1, the second loss function L2, and the third loss function L3.

2. The CT tumor image segmentation method based on a multi-attention U-shaped network according to claim 1, characterized in that, The Unet Backbone module is constructed as follows: it has a five-layer structure. The first four layers include two convolutional modules and one pooling module, and the last layer includes two convolutional layers. The output of the first layer is used as the input of the second layer and the input of visual attention 1. The output of the second layer is used as the input of the third layer and the input of visual attention 2. The output of the third layer is used as the input of the fourth layer and the input of visual attention 3. The output of the fourth layer is used as the input of the fifth layer and the input of visual attention 4. The output of the fifth layer is used as the input of visual attention 5 and the output of the overall structure. The first to fourth convolutional modules each consist of two 3×3 convolutional layers and one 2×2 max pooling layer connected in sequence. The fifth convolutional module consists of two 3×3 convolutional layers connected in sequence. The visual attention module includes visual attention 1, visual attention 2, visual attention 3, visual attention 4, and visual attention 5.

3. The CT tumor image segmentation method based on a multi-attention U-shaped network according to claim 1, characterized in that, In the blending attention module, the three convolutional kernels used for concatenation are 1*1, 3*3, and 3*3 in size, respectively; the convolutional kernels in the self-attention of the BA layer are 1*1, 3*3, and 3*3 in size, and the convolutional kernel for processing concatenation is 1*1 in size; the convolutional kernel for final processing is 1*1 in size.

4. The CT tumor image segmentation method based on a multi-attention U-shaped network according to claim 1, characterized in that, The process of constructing the loss function for the DAU-Net network model is as follows: First, the DICE coefficient is used to construct the first loss function L1 for downsampling, as shown in the following formula: Second, the DICE coefficient is used to construct the second loss function L2 for downsampling, as shown in the following formula: Third, the DICE coefficients are used to construct the third loss function L3 for downsampling, as shown in the following formula: in, This represents the output image of the Unet Backbone module. This represents the output image of the visual attention module, where Y represents the gold standard for segmentation. This represents the output image of the blending attention module.

5. The CT tumor image segmentation method based on a multi-attention U-shaped network according to claim 1, characterized in that, Visualize tumor segmentation using PySimpleGUI.

6. A system employing the CT tumor image segmentation method based on a multi-attention U-shaped network as described in any one of claims 1 to 5, characterized in that, include: A model building module, which is used to build a U-shaped network model with a multi-attention mechanism based on the UNet network; A loss function construction module, which is used to construct a loss function for the U-shaped network model based on the U-shaped network model; The image segmentation module is used to input CT images into a U-shaped network model and output segmented CT images to achieve tumor segmentation in CT images.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the CT tumor image segmentation method based on a multi-attention U-shaped network as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the CT tumor image segmentation method based on a multi-attention U-shaped network as described in any one of claims 1 to 5.

Images