Medical image segmentation method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a medical image segmentation method, device and equipment and storage medium. The method comprises: acquiring a medical image to be segmented; acquiring first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2; acquiring edge features detected by each edge detection network; calling a decoding module in a decoding network to decode the first image features to obtain initial decoding features; calling an attention module in the decoding network to perform attention processing on the ith initial decoding feature based on the ith edge feature to obtain a target attention feature, wherein the decoding network comprises a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer less than n; and performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result. The embodiments of the present application can improve the accuracy of medical image segmentation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image analysis, and particularly relates to a medical image segmentation method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, various computer vision tasks such as image prediction, target detection and image segmentation have emerged.

[0003] At present, image segmentation technology based on artificial intelligence can be applied to the medical field. For example, image segmentation technology can be used to perform image segmentation on a face image, thereby improving the accuracy and efficiency of face diagnosis. When performing image segmentation, a medical image is input into a pre-trained image segmentation model, and the medical image is segmented based on the image segmentation model to obtain an image segmentation result. However, the traditional image segmentation model does not fully extract the features of the medical image, resulting in insufficient use of feature information of the medical image and low accuracy of image segmentation. SUMMARY

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide a medical image segmentation method, device, equipment and storage medium, which can improve the accuracy of medical image segmentation.

[0006] To achieve the above-mentioned purpose, the first aspect of the embodiments of the present application provides a medical image segmentation method, comprising: acquiring a medical image to be segmented; calling n local feature extraction networks to perform cascaded feature extraction on the medical image to obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2; calling n-1 edge detection networks to perform cascaded edge detection on the medical image to obtain edge features detected by each edge detection network; inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, and calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer less than n; and performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result.

[0007] In some embodiments, the attention module comprises a channel attention layer and a spatial attention layer, and the calling the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature comprises: calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a channel attention feature; performing attention processing on the i-th initial decoding feature based on the channel attention feature to obtain an initial target attention feature; calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a spatial attention feature; and performing attention processing on the initial target attention feature based on the spatial attention feature to obtain a target attention feature.

[0008] In some embodiments, the channel attention layer comprises a max-pooling layer and an average-pooling layer, and the calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a channel attention feature comprises: for the channel attention layer in the attention module corresponding to the i-th edge detection network, calling the max-pooling layer in the channel attention layer to perform max-pooling on the i-th edge feature to obtain a first channel attention sub-feature, and calling the average-pooling layer in the channel attention layer to perform average-pooling on the i-th edge feature to obtain a second channel attention sub-feature; and adding the first channel attention sub-feature and the second channel attention sub-feature to obtain the channel attention feature.

[0009] In some embodiments, the calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a spatial attention feature comprises: calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to perform max-value operation on the i-th edge feature in the channel dimension to obtain a first spatial attention sub-feature, and perform average-value operation on the i-th edge feature in the channel dimension to obtain a second spatial attention sub-feature; and adding the first spatial attention sub-feature and the second spatial attention sub-feature to determine the spatial attention feature.

[0010] In some embodiments, the decoding the first image features in the decoding network to obtain initial decoding features comprises: performing global feature extraction on an nth first image feature to obtain a second image feature; calling a decoding module corresponding to an nth local feature extraction network in the decoding network to decode the second image feature to obtain an initial decoding feature corresponding to the nth first image feature; calling a decoding module corresponding to an (n-1)th local feature extraction network in the decoding network to decode an (n-1)th first image feature based on the initial decoding feature corresponding to the nth first image feature, to obtain an initial decoding feature corresponding to the (n-1)th first image feature; and repeatedly decoding based on the initial decoding feature corresponding to the (n-1)th first image feature until initial decoding features corresponding to all the first image features are obtained.

[0011] In some embodiments, the decoding module corresponding to the (n-1)th local feature extraction network comprises an upsampling layer and an attention gate, and the calling of the decoding module corresponding to the (n-1)th local feature extraction network in the decoding network to decode the (n-1)th first image feature based on the initial decoding feature corresponding to the nth first image feature comprises: calling the upsampling layer to perform upsampling processing on the initial decoding feature corresponding to the nth first image feature to obtain a first sampling feature; calling the attention gate to perform attention processing on the first sampling feature based on the (n-1)th first image feature to obtain a first intermediate feature; and splicing the first sampling feature and the first intermediate feature to obtain the initial decoding feature corresponding to the (n-1)th first image feature.

[0012] In some embodiments, the decoding network further comprises a convolution module corresponding to each decoding module, and the image segmentation of the medical image based on each target attention feature to obtain an image segmentation result comprises: performing convolution on an nth initial decoding feature to determine a first feature map; performing convolution on each target attention feature to determine a second feature map corresponding to each target attention feature, wherein each second feature map and the first feature map have the same dimension; determining a class probability corresponding to each pixel in the medical image based on a fusion result of the first feature map and each second feature map, wherein the class probability refers to a probability value of the pixel belonging to a classification class; and performing image segmentation on the medical image based on the class probability corresponding to each pixel to obtain an image segmentation result.

[0013] To achieve the above object, a second aspect of the embodiment of the present application provides a medical image segmentation device, comprising: an acquisition module configured to acquire a medical image to be segmented; a feature extraction module configured to call n local feature extraction networks to perform cascaded feature extraction on the medical image, to obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; an edge detection module configured to call n-1 edge detection networks to perform cascaded edge detection on the medical image, to obtain edge features detected by each of the edge detection networks; an attention processing module configured to input each of the first image features and each of the edge features into a decoding network, to call a decoding module in the decoding network to decode the first image features to obtain initial decoding features, and to call an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network comprises a decoding module corresponding to each of the local feature extraction networks and an attention module corresponding to each of the edge detection networks, the i-th initial decoding feature is decoded from the i-th first image feature and the i+1-th first image feature, and i is a positive integer less than n; and a segmentation module configured to perform image segmentation on the medical image based on each of the target attention features to obtain an image segmentation result.

[0014] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the medical image segmentation method of the first aspect when executing the computer program.

[0015] To achieve the above object, a fourth aspect of the embodiment of the present application provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the medical image segmentation method of the first aspect.

[0016] The embodiments of the present application at least have the following beneficial effects: by acquiring a medical image to be segmented, calling n local feature extraction networks to perform cascaded feature extraction on the medical image, obtaining first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2, so that each first image feature respectively contains feature information of the medical image at different scales, which is beneficial to segmenting medical images of different sizes, then calling n-1 edge detection networks to perform cascaded edge detection on the medical image, obtaining edge features detected by each edge detection network, so that each edge feature respectively contains boundary contour information of the medical image at different scales, then inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image feature to obtain initial decoding features, then calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, and finally performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result, so that the network focuses more on the edge position of the medical image, can gradually enrich the edge contour information of the medical image, effectively fuses the edge features and the first image features, supplements the edge information for the first image features, improves the processing capability of the network on the edge, further optimizes the utilization efficiency of the edge features, makes the edge contour of the image segmentation result more clear, and improves the accuracy of medical image segmentation.

[0017] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and achieved by the structure particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings are included to provide a further understanding of the technical scheme of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical scheme of the present application, and do not constitute a limitation on the technical scheme of the present application.

[0019] Figure 1 An optional flowchart of a medical image segmentation method provided by the embodiments of the present application;

[0020] Figure 2 An optional flowchart of attention processing provided by the embodiments of the present application;

[0021] Figure 3 An optional flowchart of channel attention processing provided by the embodiments of the present application;

[0022] Figure 4 An optional specific flowchart of spatial attention processing provided by the embodiments of the present application;

[0023] Figure 5 An optional flowchart of continuous decoding provided for an embodiment of the present application;

[0024] Figure 6 An optional flowchart of single decoding provided for an embodiment of the present application;

[0025] Figure 7 An optional flowchart of image segmentation provided for an embodiment of the present application;

[0026] Figure 8 An optional structural diagram of a medical image segmentation network provided for an embodiment of the present application;

[0027] Figure 9 An optional structural diagram of a decoder provided for an embodiment of the present application;

[0028] Figure 10 An optional detailed flowchart of attention processing provided for an embodiment of the present application;

[0029] Figure 11 An optional structural diagram of a decoding module provided for an embodiment of the present application;

[0030] Figure 12 An optional structural diagram of a double-layer convolutional layer provided for an embodiment of the present application;

[0031] Figure 13 An optional structural diagram of an up-sampling layer provided for an embodiment of the present application;

[0032] Figure 14 An optional structural diagram of an edge detection network during training provided for an embodiment of the present application;

[0033] Figure 15 An optional flowchart of prediction by a predictor provided for an embodiment of the present application;

[0034] Figure 16 An optional structural diagram of a medical image segmentation device provided for an embodiment of the present application;

[0035] Figure 17 An optional hardware structural diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0037] It should be noted that in various specific embodiments of the present application, when it is necessary to perform relevant processing according to data related to the characteristics of the target object, such as target object attribute information or attribute information set, the permission or consent of the target object is obtained first, and the collection, use and processing of the data comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiments of the present application need to obtain target object attribute information, the separate permission or separate consent of the target object is obtained through a pop-up window or jumping to a confirmation page, and after obtaining the separate permission or separate consent of the target object, the necessary target object related data for enabling the embodiments of the present application to normally run is obtained.

[0038] In the description of the present application, the meaning of several is one or more, the meaning of multiple is more than two, greater than, less than, more than, etc. are not included in the number, and above, below, within, etc. are included in the number.

[0039] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or above figures are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0040] In order to facilitate the understanding of the technical solutions provided by the embodiments of the present application, some key terms used by the embodiments of the present application are explained first:

[0041] Artificial intelligence (Artificial Intelligence, AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision making.

[0042] Artificial intelligence technology is a comprehensive discipline involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction systems, mechatronics, etc. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation, etc.

[0043] Machine learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0044] At present, image segmentation technology based on artificial intelligence can be applied to the medical field. For example, image segmentation technology can be used to segment facial images, thereby improving the accuracy and efficiency of face diagnosis. When performing image segmentation, a medical image is input into a pre-trained image segmentation model, and the medical image is segmented based on the image segmentation model to obtain an image segmentation result. However, the traditional image segmentation model does not fully extract the features of the medical image, resulting in insufficient utilization of the feature information of the medical image and low accuracy of image segmentation.

[0045] In view of the low accuracy of medical image segmentation, the application provides a medical image segmentation method, device, equipment and storage medium. The method comprises: acquiring a medical image to be segmented; calling n local feature extraction networks to perform cascaded feature extraction on the medical image to obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2; calling n-1 edge detection networks to perform cascaded edge detection on the medical image to obtain edge features detected by each edge detection network; inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image features to obtain initial decoding features, and calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network comprises a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, i is a positive integer less than n; and performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result. According to the scheme provided in the embodiment of the application, by acquiring the medical image to be segmented, calling n local feature extraction networks to perform cascaded feature extraction on the medical image to obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2, each first image feature contains feature information of the medical image at different scales, which is beneficial to segmenting medical images of different sizes, then calling n-1 edge detection networks to perform cascaded edge detection on the medical image to obtain edge features detected by each edge detection network, so that each edge feature contains boundary contour information of the medical image at different scales, then inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image features to obtain initial decoding features, then calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, finally performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result, so that the network focuses more on the edge position of the medical image, the edge contour information of the medical image can be gradually enriched, the edge features and the first image features are effectively fused, the edge information is supplemented for the first image features, the processing capability of the network for the edge is improved, the utilization efficiency of the edge features is further optimized, the edge contour of the image segmentation result is more clear, and the accuracy of medical image segmentation is improved.

[0046] The medical image segmentation method, device, equipment and storage medium provided in the embodiments of the application are described as follows. First, the medical image segmentation method in the embodiments of the application is described.

[0047] The medical image segmentation method provided in the embodiments of the present application relates to the technical field of computers. The medical image segmentation method provided in the embodiments of the present application can be applied to a terminal, can be applied to a server, and can also be software running in the terminal or the server. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like; the server can be configured as a stand-alone physical server, can be configured as a server cluster or a distributed system formed by multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms; and the software can be an application that implements the medical image segmentation method, but is not limited to the above forms.

[0048] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0049] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0050] As Figure 1 indicated, Figure 1 An optional flowchart of the medical image segmentation method provided in the embodiments of the present application is shown in the figure. The medical image segmentation method can be executed by a server, or can be executed by a terminal, or can be executed by a server in cooperation with a terminal. The medical image segmentation method includes but is not limited to the following steps S110 to S150:

[0051] Step S110: Obtain a medical image to be segmented.

[0052] Step S120: Call n local feature extraction networks to perform cascaded feature extraction on the medical image, to obtain first image features extracted by each local feature extraction network.

[0053] Wherein, n is a positive integer greater than 2.

[0054] The medical image can be a face image to be diagnosed, or an image of other tissues or organs of a human body, which is not limited in the embodiments of the present disclosure.

[0055] The local feature extraction network can be a ResNet network, and the version of the ResNet can be ResNetV1, ResNetV2, etc. Specifically, the local feature extraction network can be ResNet32 or ResNet50, which is not limited in the embodiments of the present disclosure.

[0056] It should be noted that the first local feature extraction network is used to receive a medical image to be segmented, and to extract features of the medical image to obtain a first image feature. The input of other local feature extraction networks is the output of the previous local feature extraction network.

[0057] It can be understood that through multi-level local feature extraction, rich feature information of the medical image at different scales can be extracted to enhance the network's ability to capture features at different scales, which helps to improve the robustness of image segmentation.

[0058] In step S130, n-1 edge detection networks are called to perform cascaded edge detection on the medical image to obtain edge features detected by each edge detection network.

[0059] Each edge detection network includes a first feature extraction module and an edge extraction module. The first feature extraction module is used to extract features of the input image to obtain an intermediate feature map. The edge extraction module is used to extract edge information from the intermediate feature map to obtain edge features. For example, Res2Net can be used to implement the first feature extraction module, and the Sobel operator can be used to implement the edge extraction module. Specifically, the input image is input into the Res2Net module for feature extraction to obtain an intermediate feature map, and then the Sobel operator is used to traverse the intermediate feature map to extract edge information to obtain edge features corresponding to the input image.

[0060] It should be noted that through Res2Net, features at different scales can be obtained, and then the Sobel operator is used to extract edge information. The edge features supplement edge information for the network through the attention mechanism, which increases the accuracy of the network in predicting edge positions. For example, through Res2Net, three features at different scales can be obtained l represents the number of stages. The Sobel operator is used to extract edge information from the features to obtain three edge features at different scales. The above process can be described by the following formula:

[0061] F l = EncoderBlock l (x)

[0062]

[0063] Sobel(x)=K X (x)+K Y (x)

[0064]

[0065]

[0066] wherein, F l is an intermediate feature map, l represents a layer number, Encoderblock l (·) represents a Res2Net encoding module at the lth layer, x is an i-th edge feature, Sobel(·) represents a Sobel operator, K X (·) represents a convolution operation in a horizontal direction, K X is a convolution kernel corresponding to the horizontal convolution operation, K Y (·) represents a convolution operation in a vertical direction, K Y is a convolution kernel corresponding to the vertical convolution operation, is an operation result of the Sobel operator, F e is a final edge feature.

[0067] It should be noted that the input of the first edge detection network is a medical image to be segmented, and the input of the other edge detection networks is an intermediate feature map output by the first feature extraction network in the previous edge detection network, realizing multi-scale feature extraction, and realizing multi-scale edge feature extraction in combination with the edge extraction module.

[0068] It can be understood that through multi-scale edge feature extraction, feature learning on the boundary is realized, features rich in boundary information are extracted, the problem of lack of boundary learning ability of the segmentation network is made up, and the segmentation ability of the prediction network can be improved.

[0069] In step S140, each first image feature and each edge feature is input to a decoding network, a decoding module in the decoding network is called to decode the first image feature to obtain an initial decoding feature, and an attention module in the decoding network is called to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature.

[0070] wherein, the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer less than n.

[0071] When i is equal to n-1, the i-th initial decoding feature is decoded from the i-th first image feature and the i+1-th initial decoding feature; when i is less than n-1, the i-th initial decoding feature is decoded from the i-th first image feature and the target attention feature corresponding to the i+1-th initial decoding feature.

[0072] It should be noted that the edge detection network is used to extract features rich in boundary information, to make up for the problem of the segmentation network lacking boundary learning ability. Specifically, first, a convolutional network is used to extract features of different scales from the input medical image. The multi-scale network structure can help the network to segment input images of different sizes. Then, the Sobel operator is used to extract boundary information from the multi-scale features. Subsequently, the attention structure is used to fuse the multi-scale boundary features and the backbone network features, so as to effectively utilize the edge features.

[0073] It can be understood that the decoding of the first image features by the decoding network is progressive, and each time a decoding module is passed, one more information contained in the first image features of different scales can be aggregated in the initial decoding features of the decoding output, so that the first image features of different scales can be fully utilized. In addition, each attention module utilizes the rich edge information output by the edge detection network at different scales, so that the network can focus on the edge contour of the image and perform multi-level edge attention processing, thereby improving the accuracy and robustness of the network in image segmentation.

[0074] In step S150, the medical image is segmented based on each target attention feature to obtain an image segmentation result.

[0075] The image segmentation result can be in the form of applying different colors to different regions of the medical image to identify the image segmentation result, so that the image segmentation result is easier to understand.

[0076] Based on this, by acquiring a medical image to be segmented, calling n local feature extraction networks for cascading feature extraction of the medical image, first image features extracted by each local feature extraction network are obtained, wherein n is a positive integer greater than 2, so that each first image feature respectively contains feature information of the medical image at different scales, which is beneficial to segmenting medical images of different sizes, then calling n-1 edge detection networks for cascading edge detection of the medical image, edge features detected by each edge detection network are obtained, so that each edge feature respectively contains boundary contour information of the medical image at different scales, then inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image feature to obtain initial decoding features, then calling an attention module in the decoding network to perform attention processing on the i th initial decoding feature based on the i th edge feature to obtain a target attention feature, and finally performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result, so that the network focuses more on the edge position of the medical image, can gradually enrich the edge contour information of the medical image, effectively fuses the edge features and the first image features, supplements the edge information for the first image features, improves the processing capability of the network on the edge, further optimizes the utilization efficiency of the edge features, makes the edge contour of the image segmentation result more clear, and improves the accuracy of medical image segmentation.

[0077] In addition, with reference to Figure 2 In an embodiment, the attention module includes a channel attention layer and a spatial attention layer, and calling the attention module in the decoding network to perform attention processing on the i th initial decoding feature based on the i th edge feature to obtain a target attention feature includes but is not limited to the following steps:

[0078] Step S210, calling the channel attention layer in the attention module corresponding to the i th edge detection network to map the i th edge feature to obtain a channel attention feature.

[0079] Step S220, performing attention processing on the i th initial decoding feature based on the channel attention feature to obtain an initial target attention feature.

[0080] Step S230, calling the spatial attention layer in the attention module corresponding to the i th edge detection network to map the i th edge feature to obtain a spatial attention feature.

[0081] Step S240, performing attention processing on the initial target attention feature based on the spatial attention feature to obtain a target attention feature.

[0082] The channel attention feature is multiplied with the i-th initial decoding feature to obtain an initial target attention feature, and then the spatial attention feature is multiplied with the initial target attention feature to obtain the target attention feature.

[0083] The target attention feature can be expressed by a formula as follows:

[0084] F(x) = F map2 · F map1 · x

[0085] F(x) is the target attention feature, F map1 is the channel attention feature, F map2 is the spatial attention feature, and x is the i-th edge feature.

[0086] Therefore, the attention module corresponding to the i-th edge detection network can effectively strengthen the edge feature through the spatial attention layer and the channel attention layer. The channel attention layer can locate the channel with rich information in the feature and enhance the weight of the channel. The spatial attention layer can focus on the position of the edge in the edge feature and strengthen the edge feature, thereby enhancing the attention of the segmentation network to the edge and the boundary position, and improving the effectiveness and robustness of the network.

[0087] In addition, referring to Figure 3 In an embodiment, the channel attention layer includes a max-pooling layer and an average-pooling layer. The channel attention layer in the attention module corresponding to the i-th edge detection network is called to map the i-th edge feature to obtain the channel attention feature, including but not limited to the following steps:

[0088] In step S310, for the channel attention layer in the attention module corresponding to the i-th edge detection network, the max-pooling layer in the channel attention layer is called to perform max-pooling on the i-th edge feature to obtain a first channel attention sub-feature, and the average-pooling layer in the channel attention layer is called to perform average-pooling on the i-th edge feature to obtain a second channel attention sub-feature.

[0089] In step S320, the first channel attention sub-feature and the second channel attention sub-feature are added to obtain the channel attention feature.

[0090] The channel attention feature can be expressed by a formula as follows:

[0091] F e1 (x) = Conv(σ2(Conv(σ1(x))))

[0092] F e2 (x) = Conv (σ2 (σ3 (x)))

[0093] F map1 = Sigmoid (F e1 +F e2 )

[0094] where F map1 is the channel attention feature, F e1 (x) is the first channel attention sub-feature, F e2 (x) is the second channel attention sub-feature, x is the ith edge feature, and x ∈ R C×H×W , that is, x is a matrix with height H, width W and channel number C, σ1 (·) is adaptive average pooling, σ2 (·) is activation function Relu, σ3 (·) is adaptive maximum pooling, Conv (·) is convolution operation, and Sigmoid (·) is normalization operation.

[0095] where, with reference to Figure 10, for the channel attention layer in the attention module corresponding to the ith edge detection network, the operation of adaptive maximum pooling can be to call the maximum pooling layer in the channel attention layer to maximum pool the ith edge feature to obtain the first channel attention sub-feature, and the operation of adaptive average pooling can be to call the maximum pooling layer in the channel attention layer to average pool the ith edge feature to obtain the second channel attention sub-feature.

[0096] where, with reference to Figure 10 , after obtaining the first channel attention sub-feature and the second channel attention sub-feature, the first channel attention sub-feature is input into the first convolution layer for convolution to complete feature extraction and obtain the third channel attention sub-feature, the third channel attention sub-feature is then input into the first activation layer for activation to obtain the fourth channel attention sub-feature, and finally the fourth channel attention sub-feature is input into the second convolution layer for convolution to obtain the fifth channel attention sub-feature; similarly, the second channel attention sub-feature is input into the third convolution layer for convolution to complete feature extraction and obtain the sixth channel attention sub-feature, the sixth channel attention sub-feature is then input into the second activation layer for activation to obtain the seventh channel attention sub-feature, and finally the seventh channel attention sub-feature is input into the fourth convolution layer for convolution to obtain the eighth channel attention sub-feature, and finally the fifth channel attention sub-feature and the eighth channel attention sub-feature are added element-wise and then normalized to obtain the channel attention.

[0097] The first convolutional layer, the second convolutional layer, the third convolutional layer and the fourth convolutional layer can all be 3*3 convolutional layers, the activation function of the first activation layer and the second activation layer can be Relu, and the normalization operation can be completed by a Sigmoid function, which is not limited in the embodiments of the present application.

[0098] Based on this, the channel attention obtains features with different performances through two branches of maximum pooling and average pooling, fuses the two features by addition, obtains a channel attention map, enriches the information of the edge feature, and can locate the channel with rich information in the edge feature and enhance the weight of the channel.

[0099] In addition, with reference to Figure 4 In an embodiment, the spatial attention layer in the attention module corresponding to the i-th edge detection network is called to map the i-th edge feature, to obtain a spatial attention feature, including but not limited to the following steps:

[0100] In step S410, the spatial attention layer in the attention module corresponding to the i-th edge detection network is called to perform maximum value operation on the i-th edge feature in the channel dimension, to obtain a first spatial attention sub-feature, and perform average value operation on the i-th edge feature in the channel dimension, to obtain a second spatial attention sub-feature.

[0101] In step S420, the first spatial attention sub-feature and the second spatial attention sub-feature are added based on the first spatial attention sub-feature and the second spatial attention sub-feature, to determine the spatial attention feature.

[0102] The maximum value operation on the i-th edge feature in the channel dimension is specifically: for each pixel position of the i-th edge feature, the maximum value of all channel maps of the i-th edge feature at the same pixel position is determined, and a new feature map composed of the maximum values corresponding to each pixel position is taken as the first spatial attention sub-feature; similarly, for each pixel position of the i-th edge feature, the average value of all channel maps of the i-th edge feature at the same pixel position is determined, and a new feature map composed of the average values corresponding to each pixel position is taken as the second spatial attention sub-feature.

[0103] With reference to Figure 10 The channel operation 1 is to call the spatial attention layer in the attention module corresponding to the i-th edge detection network to perform maximum value operation on the i-th edge feature in the channel dimension, to obtain the first spatial attention sub-feature, the channel operation 2 is to perform average value operation on the i-th edge feature in the channel dimension, to obtain the second spatial attention sub-feature, and the first spatial attention sub-feature and the second spatial attention sub-feature are added element by element, and then input into the fifth convolutional layer for convolution, to obtain the third spatial attention sub-feature, and the third spatial attention sub-feature is normalized to obtain the spatial attention feature.

[0104] The fifth convolutional layer can be a 7*7 convolutional layer, which is not limited in the embodiments of the present application.

[0105] The spatial attention feature can be expressed by a formula as follows:

[0106] F map2 (x)=Sigmoid(Conv(β1(x)+β2(x)))

[0107] The spatial attention feature F(x) is obtained by the maximum value and the average value in the channel dimension, and the first spatial attention sub-feature and the second spatial attention sub-feature are obtained by the maximum value and the average value in the channel dimension. map2 The spatial attention feature F(x) is obtained by the maximum value and the average value in the channel dimension, and the first spatial attention sub-feature and the second spatial attention sub-feature are obtained by the maximum value and the average value in the channel dimension.

[0108] Based on this, the spatial attention layer obtains the first spatial attention sub-feature and the second spatial attention sub-feature by processing the maximum value and the average value in the channel dimension, and obtains the spatial attention feature by fusing the first spatial attention sub-feature and the second spatial attention sub-feature through convolution and then performing normalization. The spatial attention feature can enhance the edge information of the first image feature, focus on the edge position in the feature, enhance the attention of the network to the edge and boundary position, and improve the effectiveness and robustness of the network.

[0109] In addition, with reference to Figure 5 In an embodiment, the decoding module in the decoding network is called to decode the first image feature to obtain the initial decoding feature, including but not limited to the following steps:

[0110] In step S510, the global feature of the nth first image feature is extracted to obtain the second image feature.

[0111] In step S520, the decoding module corresponding to the nth local feature extraction network in the decoding network is called to decode the second image feature to obtain the initial decoding feature corresponding to the nth first image feature.

[0112] In step S530, based on the initial decoding feature corresponding to the nth first image feature, the decoding module corresponding to the (n-1)th local feature extraction network in the decoding network is called to decode the (n-1)th first image feature to obtain the initial decoding feature corresponding to the (n-1)th first image feature.

[0113] In step S540, the initial decoding feature corresponding to the (n-1)th first image feature is decoded again until the initial decoding features corresponding to the first image features are obtained.

[0114] It should be noted that the Transformer is a global structure network, and the Transformer is a global structure for learning the entire picture, and the global feature extraction of the nth first image feature can be completed through the Transformer network.

[0115] The decoding module corresponding to the nth local feature extraction network includes a sixth convolutional layer. The second image feature is input into the sixth convolutional layer for convolution, which can realize decoding of the first image feature to obtain the initial decoding feature corresponding to the nth first image feature.

[0116] The input of the decoding module corresponding to the jth local feature extraction network includes the jth first image feature and the target attention feature output by the attention module corresponding to the j+1th edge detection network, that is, the input of the decoding module corresponding to the jth local feature extraction network includes the jth first image feature and the target attention feature corresponding to the j+1th edge feature. Therefore, based on the target attention feature corresponding to the j+1th edge feature, the decoding module corresponding to the jth local feature extraction network is called to decode the jth first image feature to obtain the initial decoding feature corresponding to the jth first image feature, j is greater than or equal to 1 and less than or equal to n-2. For example, when n is 4, the input of the decoding module corresponding to the first local feature extraction network includes the first first image feature and the target attention feature output by the attention module corresponding to the second edge detection network, the input of the decoding module corresponding to the second local feature extraction network includes the second first image feature and the target attention feature output by the attention module corresponding to the third edge detection network, the input of the decoding module corresponding to the third local feature extraction network includes the third first image feature and the initial decoding feature output by the fourth decoding module, and the input of the decoding module corresponding to the fourth local feature extraction network includes the second image feature.

[0117] Based on this, the global feature extraction is first performed on the nth first image feature belonging to the local feature, which can increase the global learning ability of the network. Then the decoding network starts to decode the second image feature, until the initial decoding features output by each decoding module in the decoding network and the target attention features output by each attention module are obtained, which can enhance the boundary learning ability of the network model, enhance the attention of the segmentation network to the boundary position, and improve the accuracy of medical image segmentation.

[0118] In addition, with reference to Figure 6In an embodiment, the decoding module corresponding to the (n-1)th local feature extraction network comprises an upsampling layer and an attention gate. Based on the initial decoding feature corresponding to the nth first image feature, the decoding module corresponding to the (n-1)th local feature extraction network in the decoding network is called to decode the (n-1)th first image feature to obtain the initial decoding feature corresponding to the (n-1)th first image feature, including but not limited to the following steps:

[0119] In step S610, the initial decoding feature corresponding to the nth first image feature is processed by the upsampling layer to obtain a first sampling feature.

[0120] In step S620, the attention gate is called to perform attention processing on the first sampling feature based on the (n-1)th first image feature to obtain a first intermediate feature.

[0121] In step S630, the first sampling feature and the first intermediate feature are spliced to obtain the initial decoding feature corresponding to the (n-1)th first image feature.

[0122] Wherein, the initial decoding feature corresponding to the nth first image feature is convolved by the upsampling layer, which can be bilinear interpolation, adaptive upsampling or nearest neighbor interpolation, and finally the size of the initial decoding feature is restored to be consistent with the size of the (n-1)th first image feature, which is not limited in the embodiment of the application.

[0123] Wherein, the attention processing of the attention gate on the first sampling feature can be weighting the first sampling feature based on the (n-1)th first image feature to improve the weight of the similar part of the first sampling feature, which is not limited in the embodiment of the application.

[0124] It should be noted that the structure of the decoding module corresponding to the jth local feature extraction network is consistent with the structure of the decoding module corresponding to the (n-1)th local feature extraction network, j is greater than or equal to 1, and j is less than or equal to n-2.

[0125] Wherein, reference Figure 11 The upper layer input can be the initial decoding feature corresponding to the nth first image feature, or the target attention feature corresponding to the ith edge feature. The upsampling convolution layer corresponds to the upsampling layer of the application, A is the attention gate, and C is the splicing operation. For example, the initial decoding feature corresponding to the nth first image feature is convolved by the upsampling layer to obtain a first sampling feature, the attention gate is called to perform attention processing on the first sampling feature based on the (n-1)th first image feature to obtain a first intermediate feature, and the first sampling feature and the first intermediate feature are spliced to obtain the initial decoding feature corresponding to the (n-1)th first image feature.

[0126] It should be noted that referenceFigure 13 The upsampling layer includes an upsampling operation layer, a seventh convolutional layer, an activation layer using a Relu function, and a normalization operation layer, and the input of the upsampling layer is sequentially processed by the upsampling operation layer, the seventh convolutional layer, the activation layer using the Relu function, and the normalization operation layer, for example, the initial decoding feature corresponding to the nth first image feature is sequentially processed by the upsampling operation layer, the seventh convolutional layer, the activation layer using the Relu function, and the normalization operation layer, and finally output is the first sampling feature.

[0127] Therefore, the upsampling layer is called to perform upsampling processing on the initial decoding feature corresponding to the nth first image feature to obtain the first sampling feature, which can restore the size of the initial decoding feature to be consistent with the size of the (n-1)th first image feature, facilitating subsequent attention processing and splicing operation, then the attention gate is called to perform attention processing on the first sampling feature based on the (n-1)th first image feature to obtain the first intermediate feature, which can make the network pay more attention to different size feature information, and finally the first sampling feature and the first intermediate feature are spliced to obtain the initial decoding feature corresponding to the (n-1)th first image feature, realizing the fusion of multi-scale features, so that the initial decoding feature has rich multi-scale feature information, which is beneficial to improve the accuracy and robustness of medical image segmentation.

[0128] In addition, with reference to Figure 7 In an embodiment, the decoding network further includes a convolutional module corresponding to each decoding module, and the medical image is segmented based on each target attention feature to obtain an image segmentation result, including but not limited to the following steps:

[0129] Step S710, convolving the nth initial decoding feature to determine a first feature map.

[0130] Step S720, respectively convolving each target attention feature to determine a second feature map corresponding to each target attention feature.

[0131] Step S730, determining a class probability corresponding to each pixel in the medical image based on the fusion result of the first feature map and each second feature map.

[0132] Step S740, segmenting the medical image based on the class probability corresponding to each pixel to obtain an image segmentation result.

[0133] Wherein, the dimensions of each second feature map and the first feature map are the same.

[0134] Wherein, the class probability refers to the probability value of the pixel belonging to the classification category.

[0135] Wherein, since the decoding network further comprises the convolution modules corresponding to the decoding modules, there are totally n convolution modules, the input of the nth convolution module is the nth initial decoding feature, and the input of the ith convolution module is the ith target attention feature.

[0136] Wherein, the convolution module corresponding to each decoding module can comprise a double-layer convolution layer, which is not limited in the embodiments of the present application.

[0137] It should be noted that the first feature map and each second feature map correspond to a possible image segmentation result, and the first feature map and each second feature map can be fused to obtain a fusion result of the first feature map and each second feature map, which can provide a more comprehensive basis for medical image segmentation and correspond to a more accurate image segmentation result.

[0138] Wherein, the first feature map and each second feature map can be added to obtain a fusion result, and the fusion result can be normalized, that is, each pixel of the fusion result has a set of probability distributions corresponding to each category, the category corresponding to the maximum probability in the probability distribution of each pixel is taken as the target category, and finally pixels with the same target category can be divided by the same mark to obtain an image segmentation result. For example, the categories include forehead, left face, right face, nose and chin, and a certain pixel of the fusion result has a corresponding probability distribution: the corresponding probability of forehead is 0.2, the corresponding probability of left face is 0.3, the corresponding probability of right face and nose is 0.1, and the corresponding probability of chin is 0.4. The corresponding probability of chin is the largest, so the chin is the target category corresponding to the above-mentioned pixel. If the mark corresponding to the chin is blue, the above-mentioned pixel can be marked as blue, which realizes the division of the above-mentioned pixel into the category of chin.

[0139] Wherein, the fusion result can be normalized by a softmax function and by an argmax function, which is not limited in the embodiments of the present application.

[0140] Based on this, the first feature map can provide relatively global image information of the medical image, and each second feature map contains edge information of different scales and different weights of the medical image. Therefore, each second feature map can help the network better identify the edges and boundaries in the medical image, so that the fusion result of the first feature map and each second feature map has relatively global image edge information, and the category probability corresponding to each pixel in the medical image is relatively accurate, thereby improving the accuracy of the image segmentation result.

[0141] It should be noted that reference Figure 8, the input image is a medical image, the encoder is composed of a plurality of local feature extraction networks, the local feature extraction network is realized based on a ResNetV2 module, the edge network is composed of a plurality of edge detection networks, the edge detection network is realized based on a Res2Net module and a Sobel operator, the first feature extraction module of the edge detection network is realized based on a Res2Net module, and the edge extraction module of the edge detection network is realized based on a Sobel operator, the Transformer is a global feature extraction network, and the decoder is a decoding network; first, the medical image is input into the encoder, the encoder includes four local feature extraction networks, the four local feature extraction networks are used to perform cascaded feature extraction on the input medical image, four first image features with decreasing scales are obtained, the first three first image features are input into the first three stages of the decoder respectively, the fourth first image feature is input into the global feature extraction network for global feature extraction, and a second image feature is obtained; then, the second image feature is input into the fourth stage of the decoder, and the medical image is input into the edge network; the edge network includes three edge detection networks, the first feature extraction module of each edge detection network is used to perform cascaded feature extraction on the medical image, intermediate feature maps corresponding to the edge detection networks are obtained, the intermediate feature maps are input into the edge extraction module of each edge detection network for edge extraction, edge features corresponding to the edge detection networks are obtained, and the edge features corresponding to the three edge detection networks are input into the first three stages of the decoder respectively; the decoder performs decoding and attention processing on the plurality of first image features and the plurality of edge features, obtains a first feature map corresponding to the fourth first image feature, obtains second feature maps corresponding to the first three first image features, and the three second feature maps correspond to the three edge features respectively; finally, the first feature map and the second feature maps are fused to obtain a fusion result, the medical image is subjected to image segmentation based on the fusion result, and an image segmentation result, i.e., an output image, is obtained.

[0142] It should be noted that, with reference to Figure 14 In the training process, the edge detection network is also provided with a predictor, the input of the predictor is the output of the edge extraction module, i.e., the edge feature, with reference to Figure 15 The edge feature is input into the predictor, and the edge feature sequentially undergoes convolution, activation, standardization, convolution, activation, standardization and up-sampling operations, and finally the output feature, i.e., the prediction feature, is obtained; in addition, the loss function of the edge detection network is a Lovasz loss function, which can be represented by the following formula:

[0143]

[0144] Wherein, Loss Lovase is the model loss, y is the prediction feature, is the true label.

[0145] It can be understood that, since the predictor can improve the integrity of the edge detection network and introduce the Lovasz loss function to determine the model loss, it can help the network learn and accelerate the network convergence, improve the training efficiency and training effect of the network.

[0146] It should be noted that, with reference to Figure 9 , Figure 9 An optional structure schematic diagram of the decoder provided by the embodiment of the present application is shown in FIG. 6. The encoder has four stages, and each stage has different inputs. The decoding structure corresponds to the decoding module of the present application, and the edge attention structure corresponds to the attention module of the present application. For example, the fourth stage of the encoder is provided with a sixth convolutional layer and a double-layer convolutional layer. The input of the sixth convolutional layer is the second image feature, and the output is the initial decoding feature. The initial encoding is input into the double-layer convolutional layer of the fourth stage to obtain the first feature map. The first to third stages of the encoder are all provided with a decoding module, an attention module and a double-layer convolutional layer. The input of the decoding module of the third stage is the third first image feature and the initial decoding feature of the fourth stage. The input of the decoding module of the second stage is the second first image feature and the initial decoding feature of the third stage. The input of the decoding module of the first stage is the first first image feature and the initial decoding feature of the second stage. In addition, the input of the attention module of the third stage is the third edge feature and the initial decoding feature of the third stage. The input of the attention module of the second stage is the second edge feature and the initial decoding feature of the second stage. The input of the attention module of the first stage is the first edge feature and the initial decoding feature of the first stage. The outputs of the double-layer convolutional layers of the third, second and first stages are all second feature maps.

[0147] It is worth noting that, during training, the model sub-loss can be determined based on the output of each double-layer convolutional layer. For example, L Seg1 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the first stage of the decoding network, L Seg2 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the second stage of the decoding network, L Seg3 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the third stage of the decoding network, and L Seg4 is the model sub-loss determined based on the first feature map output by the double-layer convolutional layer of the fourth stage of the decoding network.

[0148] Specifically, the model sub-loss can be expressed by the following formula:

[0149]

[0150]

[0151] LSeg = aL Dice + bL CE

[0152] wherein, L CE is a cross-entropy loss function, L Dice is a Dice loss function, L Seg is a segmentation loss function, L Seg is used to determine the model sub-loss, N is the total number of pixels in a medical image, is a predicted segmentation map, yi is an original segmentation map, and a and b are both coefficients, and a and b are set to 0.7 and 0.3 respectively.

[0153] It should be noted that the model loss of the entire network can be represented by the following formula:

[0154] Loss = L Seg1 + L Seg2 + L Seg3 + L Seg4

[0155] wherein, Loss is the model loss of the entire network, L Seg1 , L Seg2 , L Seg3 and L Seg4 are the model sub-losses corresponding to the encoder stages 1, 2, 3 and 4 respectively.

[0156] It should be noted that, referring to Figure 12 , the double-layer convolutional layer includes two convolutional structures connected in series, and the convolutional structure includes an eighth convolutional layer, an activation layer using a Relu function and a standardization operation layer.

[0157] In addition, referring to Figure 16 , the present application also provides a medical image segmentation device 1600, comprising:

[0158] An acquisition module 1610 is configured to acquire a medical image to be segmented;

[0159] A feature extraction module 1620 is configured to call n local feature extraction networks to perform cascaded feature extraction on the medical image to obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2.

[0160] An edge detection module 1630 is configured to call n-1 edge detection networks to perform cascaded edge detection on the medical image to obtain edge features detected by each edge detection network.

[0161] The attention processing module 1640 is configured to input each first image feature and each edge feature into a decoding network, call a decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, and call an attention module in the decoding network to perform attention processing on the i th initial decoding feature based on the i th edge feature to obtain a target attention feature, where the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, the i th initial decoding feature is obtained by decoding the i th first image feature and the (i+1) th first image feature, and i is a positive integer less than n.

[0162] The segmentation module 1650 is configured to perform image segmentation on the medical image based on each target attention feature to obtain an image segmentation result.

[0163] It can be understood that the specific implementation of the medical image segmentation apparatus 1600 is basically the same as the specific embodiments of the medical image segmentation method described above, and thus will not be described herein again.

[0164] In addition, referring to Figure 17 , Figure 17 a hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0165] The processor 1701 can be implemented in a manner of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute a related program to implement the technical solutions provided by the embodiments of the present application.

[0166] The memory 1702 can be implemented in a form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 1702 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1702 and are called and executed by the processor 1701 to implement the medical image segmentation method of the embodiments of the present application.

[0167] The input / output interface 1703 is configured to realize information input and output.

[0168] The communication interface 1704 is configured to realize the communication interaction between the device and other devices. The communication can be realized in a wired manner (for example, a USB, a network cable, or the like) or in a wireless manner (for example, a mobile network, WIFI, Bluetooth, or the like).

[0169] The bus 1705 is configured to transmit information between various components (for example, the processor 1701, the memory 1702, the input / output interface 1703, and the communication interface 1704) of the device.

[0170] The processor 1701, the memory 1702, the input / output interface 1703, and the communication interface 1704 are connected to each other through the bus 1705 to realize the communication connection between the device.

[0171] The embodiment of the present application further provides a storage medium, which is a computer readable storage medium, is used for computer readable storage, and stores one or more programs. The one or more programs can be executed by one or more processors to realize the medical image segmentation method.

[0172] The memory is a non-transitory computer readable storage medium, and can be used to store a non-transitory software program and a non-transitory computer executable program. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, for example, at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0173] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0174] Those skilled in the art can understand that, Figures 1 to 17 The technical solutions shown in the foregoing embodiments do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than those shown in the drawings, or combine certain steps, or different steps.

[0175] The device embodiments described above are only schematic, and units described as separate components can or can not be physically separate, that is, can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to realize the purpose of the embodiments of the present application.

[0176] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0177] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of orderly or chronological permutations. Moreover, the terms "comprise", "have" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, article, or apparatus that comprises a list of steps or units can not necessarily be limited to those steps or units, but can include additional steps or units not expressly listed or inherent to such process, method, article, or apparatus.

[0178] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0179] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0181] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0182] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0183] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A medical image segmentation method characterized by, The method comprises: acquiring a medical image to be segmented; calling n local feature extraction networks to perform cascaded feature extraction on the medical image to obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; calling n-1 edge detection networks to perform cascaded edge detection on the medical image to obtain edge features detected by each of the edge detection networks; inputting each of the first image features and each of the edge features into a decoding network, calling a decoding module in the decoding network to decode the first image features to obtain initial decoding features, and calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network comprises a decoding module corresponding to each of the local feature extraction networks and an attention module corresponding to each of the edge detection networks, and i is a positive integer less than n; performing image segmentation on the medical image based on each of the target attention features to obtain an image segmentation result.

2. The medical image segmentation method according to claim 1, characterized in that, The attention module comprises a channel attention layer and a spatial attention layer, and the calling of the attention module in the decoding network based on the i-th edge feature to perform attention processing on the i-th initial decoding feature to obtain a target attention feature comprises: mapping the i-th edge feature by the channel attention layer in the attention module corresponding to the i-th edge detection network to obtain a channel attention feature; performing attention processing on the i-th initial decoding feature based on the channel attention feature to obtain an initial target attention feature; mapping the i-th edge feature by the spatial attention layer in the attention module corresponding to the i-th edge detection network to obtain a spatial attention feature; performing attention processing on the initial target attention feature based on the spatial attention feature to obtain a target attention feature.

3. The medical image segmentation method according to claim 2, characterized in that, The channel attention layer comprises a max-pooling layer and an average-pooling layer, and the mapping of the i-th edge feature by the channel attention layer in the attention module corresponding to the i-th edge detection network to obtain a channel attention feature comprises: for the channel attention layer in the attention module corresponding to the i-th edge detection network, performing max-pooling on the i-th edge feature by the max-pooling layer in the channel attention layer to obtain a first channel attention sub-feature, and performing average-pooling on the i-th edge feature by the average-pooling layer in the channel attention layer to obtain a second channel attention sub-feature; performing addition based on the first channel attention sub-feature and the second channel attention sub-feature to obtain a channel attention feature.

4. The medical image segmentation method according to claim 2, characterized in that, The mapping of the i-th edge feature by the spatial attention layer in the attention module corresponding to the i-th edge detection network to obtain a spatial attention feature comprises: The spatial attention layer in the attention module corresponding to the i-th edge detection network is called to take the maximum value of the i-th edge feature in the channel dimension to obtain a first spatial attention sub-feature, and to take the average value of the i-th edge feature in the channel dimension to obtain a second spatial attention sub-feature; The spatial attention feature is determined by adding the first spatial attention sub-feature and the second spatial attention sub-feature.

5. The medical image segmentation method according to claim 1, characterized in that, The decoding module in the decoding network is called to decode the first image feature to obtain an initial decoding feature, including: The global feature extraction is performed on the n-th first image feature to obtain a second image feature; The decoding module corresponding to the n-th local feature extraction network in the decoding network is called to decode the second image feature to obtain the initial decoding feature corresponding to the n-th first image feature; Based on the initial decoding feature corresponding to the n-th first image feature, the decoding module corresponding to the n-1-th local feature extraction network in the decoding network is called to decode the n-1-th first image feature to obtain the initial decoding feature corresponding to the n-1-th first image feature: The initial decoding feature corresponding to each first image feature is obtained by decoding the initial decoding feature corresponding to the n-1-th first image feature again.

6. The medical image segmentation method according to claim 5, characterized in that, The decoding module corresponding to the n-1-th local feature extraction network includes an upsampling layer and an attention gate, and the decoding module corresponding to the n-1-th local feature extraction network in the decoding network is called to decode the n-1-th first image feature based on the initial decoding feature corresponding to the n-th first image feature to obtain the initial decoding feature corresponding to the n-1-th first image feature, including: The upsampling layer is called to perform upsampling processing on the initial decoding feature corresponding to the n-th first image feature to obtain a first sampling feature; The attention gate is called to perform attention processing on the first sampling feature based on the n-1-th first image feature to obtain a first intermediate feature; The first sampling feature and the first intermediate feature are spliced to obtain the initial decoding feature corresponding to the n-1-th first image feature.

7. The medical image segmentation method according to claim 5, characterized in that, The decoding network further includes a convolution module corresponding to each decoding module, and the medical image is segmented based on each target attention feature to obtain an image segmentation result, including: The n-th initial decoding feature is convolved to determine a first feature map; Each target attention feature is convolved to determine a second feature map corresponding to each target attention feature, wherein the dimensions of each second feature map and the first feature map are the same; Based on the fusion result of the first feature map and each second feature map, the category probability corresponding to each pixel in the medical image is determined, wherein the category probability refers to the probability value of the pixel belonging to a classification category. Image segmentation is performed on the medical image based on the category probability corresponding to each pixel, and an image segmentation result is obtained.

8. A medical image segmentation apparatus characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire a medical image to be segmented. A feature extraction module is configured to call n local feature extraction networks to perform cascaded feature extraction on the medical image, and obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2. An edge detection module is configured to call n-1 edge detection networks to perform cascaded edge detection on the medical image, and obtain edge features detected by each edge detection network. An attention processing module is configured to input each first image feature and each edge feature into a decoding network, call a decoding module in the decoding network to decode the first image features to obtain initial decoding features, and call an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network comprises a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, the i-th initial decoding feature is decoded from the i-th first image feature and the i+1-th first image feature, and i is a positive integer less than n. A segmentation module is configured to perform image segmentation on the medical image based on each target attention feature, and obtain an image segmentation result.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the medical image segmentation method of any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized by The computer program is executed by the processor to implement the medical image segmentation method of any one of claims 1 to 7.

Images