Medical image segmentation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a medical image segmentation method and device, equipment and a storage medium. The method comprises the steps that a to-be-segmented medical image is acquired; obtaining a first image feature extracted by each local feature extraction network, n being a positive integer greater than 2; obtaining edge features detected by each edge detection network; calling a decoding module in a decoding network to decode the first image feature to obtain an initial decoding feature, and calling an attention module in the decoding network to perform attention processing on the ith initial decoding feature based on the ith edge feature to obtain a target attention feature, the decoding network comprises a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer smaller than n; and performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result. According to the embodiment of the invention, the precision of medical image segmentation can be improved.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of medical image analysis, and in particular to a medical image segmentation method, apparatus, device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, a variety of computer vision tasks have emerged, such as image prediction, target detection, and image segmentation.

[0003] At present, image segmentation technology based on artificial intelligence can be applied in the medical field. For example, image segmentation technology can be used to segment facial images, thereby improving the accuracy and efficiency of facial diagnosis. When performing image segmentation, the medical image is input into a pre-trained image segmentation model, and the medical image is segmented based on the image segmentation model to obtain the image segmentation result. However, the traditional image segmentation model does not fully extract the features of medical images, resulting in insufficient utilization of the feature information of medical images, making the image segmentation accuracy low. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide a medical image segmentation method, apparatus, device and storage medium, which can improve the accuracy of medical image segmentation.

[0006] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a medical image segmentation method, comprising: obtaining a medical image to be segmented; calling n local feature extraction networks to perform cascade feature extraction on the medical image to obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; calling n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each of the edge detection networks; inputting each of the first image features and each of the edge features into a decoding network, calling a decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each of the local feature extraction networks and an attention module corresponding to each of the edge detection networks, and i is a positive integer less than n; performing image segmentation on the medical image based on each of the target attention features to obtain an image segmentation result.

[0007] In some embodiments, the attention module includes a channel attention layer and a spatial attention layer, and calling the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature includes: calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a channel attention feature; performing attention processing on the i-th initial decoding feature based on the channel attention feature to obtain an initial target attention feature; calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a spatial attention feature; performing attention processing on the initial target attention feature based on the spatial attention feature to obtain a target attention feature.

[0008] In some embodiments, the channel attention layer includes a maximum pooling layer and an average pooling layer, and calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a channel attention feature, including: for the channel attention layer in the attention module corresponding to the i-th edge detection network, calling the maximum pooling layer in the channel attention layer to perform maximum pooling on the i-th edge feature to obtain a first channel attention sub-feature, calling the average pooling layer in the channel attention layer to perform average pooling on the i-th edge feature to obtain a second channel attention sub-feature; adding the first channel attention sub-feature and the second channel attention sub-feature to obtain a channel attention feature.

[0009] In some embodiments, the calling of the spatial attention layer in the attention module corresponding to the ith edge detection network to map the ith edge feature to obtain a spatial attention feature includes: calling the spatial attention layer in the attention module corresponding to the ith edge detection network, taking the maximum value of the ith edge feature in the channel dimension to obtain a first spatial attention sub-feature, and taking the average value of the ith edge feature in the channel dimension to obtain a second spatial attention sub-feature; determining the spatial attention feature based on the addition of the first spatial attention sub-feature and the second spatial attention sub-feature.

[0010] In some embodiments, calling the decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature includes: performing global feature extraction on the nth first image feature to obtain a second image feature; calling the decoding module corresponding to the nth local feature extraction network in the decoding network to decode the second image feature to obtain an initial decoding feature corresponding to the nth first image feature; based on the initial decoding feature corresponding to the nth first image feature, calling the decoding module corresponding to the n-1th local feature extraction network in the decoding network to decode the n-1th first image feature to obtain an initial decoding feature corresponding to the n-1th first image feature: decoding again based on the initial decoding feature corresponding to the n-1th first image feature until the initial decoding features corresponding to each of the first image features are obtained.

[0011] In some embodiments, the decoding module corresponding to the n-1th local feature extraction network includes an upsampling layer and an attention gate, and based on the initial decoding feature corresponding to the n-1th first image feature, calling the decoding module corresponding to the n-1th local feature extraction network in the decoding network to decode the n-1th first image feature to obtain the initial decoding feature corresponding to the n-1th first image feature, including: calling the upsampling layer to upsample the initial decoding feature corresponding to the n-1th first image feature to obtain a first sampling feature; calling the attention gate to perform attention processing on the first sampling feature based on the n-1th first image feature to obtain a first intermediate feature; splicing the first sampling feature and the first intermediate feature to obtain the initial decoding feature corresponding to the n-1th first image feature.

[0012] In some embodiments, the decoding network also includes a convolution module corresponding to each of the decoding modules, and the image segmentation of the medical image based on each of the target attention features to obtain an image segmentation result includes: convolving the nth initial decoding feature to determine a first feature map; convolving each of the target attention features respectively to determine a second feature map corresponding to each of the target attention features, wherein the dimensions of each of the second feature maps and the first feature map are the same; based on the fusion result of the first feature map and each of the second feature maps, determining the category probability corresponding to each pixel in the medical image, wherein the category probability refers to the probability value that the pixel belongs to a classification category; performing image segmentation on the medical image based on the category probability corresponding to each of the pixels to obtain an image segmentation result.

[0013] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a medical image segmentation device, including: an acquisition module, used to acquire a medical image to be segmented; a feature extraction module, used to call n local feature extraction networks to perform cascade feature extraction on the medical image, and obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; an edge detection module, used to call n-1 edge detection networks to perform cascade edge detection on the medical image, and obtain edge features detected by each of the edge detection networks; an attention processing module, used to input each of the first image features and each of the edge features into a decoding network, and call the The decoding module in the decoding network decodes the first image feature to obtain an initial decoding feature, and calls the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes decoding modules corresponding to each local feature extraction network and attention modules corresponding to each edge detection network, and the i-th initial decoding feature is obtained by decoding the i-th first image feature and the i+1-th first image feature, where i is a positive integer less than n; a segmentation module is used to perform image segmentation on the medical image based on each target attention feature to obtain an image segmentation result.

[0014] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the medical image segmentation method described in the first aspect is implemented.

[0015] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, the medical image segmentation method described in the first aspect is implemented.

[0016] The embodiments of the present application include at least the following beneficial effects: by acquiring a medical image to be segmented, calling n local feature extraction networks to perform cascade feature extraction on the medical image, and obtaining first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2, so that each first image feature contains feature information of the medical image at different scales, which is conducive to segmenting medical images of different sizes, and then calling n-1 edge detection networks to perform cascade edge detection on the medical image, and obtaining edge features detected by each edge detection network, so that each edge feature contains boundary contour information of the medical image at different scales, and then each first image feature and each edge feature are input into a decoding network, and the first image feature and each edge feature are input into a decoding network, and the first image feature and the ... The decoding module in the decoding network is used to decode the first image feature to obtain the initial decoding feature, and then the attention module in the decoding network is called to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain the target attention feature. Finally, the medical image is segmented based on each target attention feature to obtain the image segmentation result, so that the network can focus more on the edge position of the medical image, gradually enrich the edge contour information of the medical image, effectively integrate the edge feature with the first image feature, supplement the edge information for the first image feature, improve the network's edge processing ability, further optimize the utilization efficiency of the edge feature, make the edge contour of the image segmentation result clearer, and improve the accuracy of medical image segmentation.

[0017] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0019] Figure 1 An optional flowchart of a medical image segmentation method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of an optional flow chart of attention processing provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of an optional flow chart of channel attention processing provided in an embodiment of the present application;

[0022] Figure 4 An optional specific flow chart of spatial attention processing provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of an optional flow chart of continuous decoding provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of an optional flow chart of a single decoding provided in an embodiment of the present application;

[0025] Figure 7 A schematic diagram of an optional process of image segmentation provided in an embodiment of the present application;

[0026] Figure 8 An optional structural diagram of a medical image segmentation network provided in an embodiment of the present application;

[0027] Fig. 9 An optional structural diagram of a decoder provided in an embodiment of the present application;

[0028] Fig.10 An optional specific flow chart of attention processing provided in an embodiment of the present application;

[0029] Fig.11 An optional structural diagram of a decoding module provided in an embodiment of the present application;

[0030] Fig.12 A schematic diagram of an optional structure of a double-layer convolutional layer provided in an embodiment of the present application;

[0031] Fig.13 A schematic diagram of an optional structure of an upsampling layer provided in an embodiment of the present application;

[0032] Fig.14 An optional structural diagram of an edge detection network provided in an embodiment of the present application during training;

[0033] Fig.15 A schematic diagram of an optional flow chart for prediction by the predictor provided in an embodiment of the present application;

[0034] Fig.16 An optional structural diagram of a medical image segmentation device provided in an embodiment of the present application;

[0035] Fig.17 A schematic diagram of an optional hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0037] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain the target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page, etc., and after the separate permission or separate consent of the target object is clearly obtained, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0038] In the description of this application, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood to exclude the number, and “above”, “below”, “within”, etc. are understood to include the number.

[0039] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0040] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0041] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0042] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0043] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0044] At present, image segmentation technology based on artificial intelligence can be applied in the medical field. For example, image segmentation technology can be used to segment facial images, thereby improving the accuracy and efficiency of facial diagnosis. When performing image segmentation, the medical image is input into a pre-trained image segmentation model, and the medical image is segmented based on the image segmentation model to obtain the image segmentation result. However, the traditional image segmentation model does not fully extract the features of medical images, resulting in insufficient utilization of the feature information of medical images, making the image segmentation accuracy low.

[0045] In response to the problem of low accuracy in medical image segmentation, the present application provides a medical image segmentation method, apparatus, device and storage medium, the method comprising: obtaining a medical image to be segmented; calling n local feature extraction networks to perform cascade feature extraction on the medical image to obtain first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2; calling n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each edge detection network; inputting each first image feature and each edge feature into a decoding network, calling a decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer less than n; performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result. According to the solution provided by the embodiment of the present application, by obtaining a medical image to be segmented, calling n local feature extraction networks to perform cascade feature extraction on the medical image, and obtaining first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2, so that each first image feature contains feature information of the medical image at different scales, which is conducive to segmenting medical images of different sizes, and then calling n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each edge detection network, so that each edge feature contains boundary contour information of the medical image at different scales, and then each first image feature and each edge feature are input into the decoding network, and the decoding network is called. The decoding module in the coding network decodes the first image feature to obtain the initial decoding feature, and then calls the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain the target attention feature, and finally performs image segmentation on the medical image based on each target attention feature to obtain the image segmentation result, so that the network focuses more on the edge position of the medical image, can gradually enrich the edge contour information of the medical image, effectively integrate the edge feature with the first image feature, supplement the edge information for the first image feature, improve the network's edge processing ability, further optimize the utilization efficiency of the edge feature, make the edge contour of the image segmentation result clearer, and improve the accuracy of medical image segmentation.

[0046] The medical image segmentation method, apparatus, device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the medical image segmentation method in the embodiments of the present application is described.

[0047] The medical image segmentation method provided in the embodiment of the present application relates to the field of computer technology. The medical image segmentation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the medical image segmentation method, etc., but is not limited to the above forms.

[0048] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0049] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0050] like Figure 1 As shown, Figure 1 An optional flow chart of a medical image segmentation method provided in an embodiment of the present application, the medical image segmentation method can be executed by a server, or can also be executed by a terminal, or can also be executed by a server in conjunction with a terminal, the medical image segmentation method includes but is not limited to the following steps S110 to S150:

[0051] Step S110, obtaining a medical image to be segmented.

[0052] Step S120, calling n local feature extraction networks to perform cascade feature extraction on the medical image, and obtaining the first image features extracted by each local feature extraction network.

[0053] Wherein, n is a positive integer greater than 2.

[0054] The medical image may be a facial image to be diagnosed, or an image of other tissues or organs of the human body, which is not limited in the embodiments of the present disclosure.

[0055] Among them, the local feature extraction network can be a ResNet network, and the version of ResNet can be ResNetV1 and ResNetV2, etc. Specifically, the local feature extraction network can be ResNet32 or ResNet50, which is not limited in the embodiments of the present disclosure.

[0056] It should be noted that the first local feature extraction network is used to receive the medical image to be segmented and perform feature extraction on the medical image to obtain the first image feature. The inputs of other local feature extraction networks are all the outputs of the previous local feature extraction network.

[0057] It can be understood that through multi-level local feature extraction, rich feature information of medical images at different scales can be extracted to enhance the network's ability to capture features of different scales, which helps to improve the robustness of image segmentation.

[0058] Step S130, calling n-1 edge detection networks to perform cascade edge detection on the medical image, and obtaining edge features detected by each edge detection network.

[0059] Among them, each edge detection network includes a first feature extraction module and an edge extraction module. The first feature extraction module is used to extract features from the input image to obtain an intermediate feature map; the edge extraction module is used to extract edge information from the intermediate feature map to obtain edge features. For example, the first feature extraction module can be implemented using Res2Net, and the edge extraction module can be implemented using the Sobel operator. Specifically, the input image is input into the Res2Net module for feature extraction to obtain an intermediate feature map, and then the Sobel operator is used to traverse the intermediate feature map to extract the edge information and obtain the edge features corresponding to the input image.

[0060] It should be noted that through Res2Net, features of different scales can be obtained, and then the edge information is refined through the Sobel operator. The edge features supplement the edge information for the network through the attention mechanism, increasing the accuracy of the network's prediction of the edge position. For example, through Res2Net, three features of different scales can be obtained l represents the number of stages, and the Sobel operator is used to calculate the features. Edge information extraction is performed to obtain edge features of three different scales. The above process can be described by the following formula:

[0061] F l =EncoderBlock l (x)

[0062]

[0063] Sobel(x)=K X (x)+K Y (x)

[0064]

[0065]

[0066] Among them, F l is the intermediate feature map, l represents the number of layers, Encoderblock l (·) represents the Res2Net encoding module at layer l, x is the ith edge feature, Sobel (·) represents the Sobel operator, K X (·) represents the convolution operation in the horizontal direction, K X is the convolution kernel corresponding to the horizontal convolution operation, K Y (·) represents the convolution operation in the vertical direction, K Y is the convolution kernel corresponding to the vertical convolution operation, is the result of the Sobel operator, F e is the final edge feature.

[0067] It should be noted that the input of the first edge detection network is the medical image to be segmented, and the input of other edge detection networks are the intermediate feature maps output by the first feature extraction network in the previous edge detection network, which realizes multi-scale feature extraction. Combined with the edge extraction module, multi-scale edge feature extraction is realized.

[0068] It can be understood that through multi-scale edge feature extraction, feature learning on the boundary is realized, features rich in boundary information are extracted, and the problem that the segmentation network lacks boundary learning ability is compensated, which can improve the segmentation ability of the prediction network.

[0069] Step S140, input each first image feature and each edge feature into the decoding network, call the decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, and call the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature.

[0070] Among them, the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and i is a positive integer less than n.

[0071] Among them, when i is equal to n-1, the i-th initial decoding feature is decoded by the i-th first image feature and the i+1-th initial decoding feature; when i is less than n-1, the i-th initial decoding feature is decoded by the i-th first image feature and the target attention feature corresponding to the i+1-th initial decoding feature.

[0072] It should be noted that the edge detection network is used to extract features rich in boundary information to make up for the problem that the segmentation network lacks boundary learning ability. Specifically, the convolutional network is first used to extract features of different scales from the input medical image. The multi-scale network structure can help the network segment input images of different sizes. Subsequently, the Sobel operator is used to extract boundary information from multi-scale features. Next, the multi-scale boundary features are fused with the backbone network features through the attention structure to achieve effective utilization of edge features.

[0073] It can be understood that the decoding network decodes the first image feature layer by layer, and each time it passes through a decoding module, it can aggregate the information contained in one more first image feature of a different scale into the initial decoding feature of the decoding output, and can make full use of multiple first image features of different scales. In addition, each attention module uses the rich edge information output by the edge detection network at different scales, so that the network can focus on the edge contour of the image, perform multi-level edge attention processing, and improve the accuracy and robustness of the network for image segmentation.

[0074] Step S150, performing image segmentation on the medical image based on each target attention feature to obtain an image segmentation result.

[0075] The image segmentation result may be presented in the form of marking different regions on the medical image with different colors, so that the image segmentation result is easier to understand.

[0076] Based on this, by obtaining the medical image to be segmented, calling n local feature extraction networks to perform cascade feature extraction on the medical image, and obtaining the first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2, so that each first image feature contains the feature information of the medical image at different scales, which is conducive to segmenting medical images of different sizes, and then calling n-1 edge detection networks to perform cascade edge detection on the medical image, and obtaining the edge features detected by each edge detection network, so that each edge feature contains the boundary contour information of the medical image at different scales, and then each first image feature and each edge feature are input into the decoding network, and the decoding network is called. The decoding module decodes the first image feature to obtain an initial decoding feature, and then calls the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature. Finally, the medical image is segmented based on each target attention feature to obtain an image segmentation result, so that the network focuses more on the edge position of the medical image, can gradually enrich the edge contour information of the medical image, effectively integrate the edge feature with the first image feature, supplement the edge information for the first image feature, improve the network's edge processing capability, further optimize the utilization efficiency of the edge feature, make the edge contour of the image segmentation result clearer, and improve the accuracy of medical image segmentation.

[0077] In addition, refer to Figure 2 In one embodiment, the attention module includes a channel attention layer and a spatial attention layer, and the attention module in the decoding network is called to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain the target attention feature, including but not limited to the following steps:

[0078] Step S210, calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain the channel attention feature.

[0079] Step S220, performing attention processing on the i-th initial decoding feature based on the channel attention feature to obtain the initial target attention feature.

[0080] Step S230, calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain the spatial attention feature.

[0081] Step S240, performing attention processing on the initial target attention feature based on the spatial attention feature to obtain the target attention feature.

[0082] Among them, referring to Figure 10, the i-th initial decoding feature is subjected to attention processing based on the channel attention feature, which may be element-wise multiplication of the channel attention feature and the i-th initial decoding feature to obtain the initial target attention feature; then the initial target attention feature is subjected to attention processing based on the spatial attention feature, which may be element-wise multiplication of the spatial attention feature and the initial target attention feature to obtain the target attention feature.

[0083] Among them, the target attention feature can be expressed by the formula, the formula is as follows:

[0084] F(x)=F map2 ·F map1 ·x

[0085] Among them, F(x) is the target attention feature, F map1 is the channel attention feature, F map2 is the spatial attention feature, and x is the i-th edge feature.

[0086] Based on this, the attention module corresponding to the i-th edge detection network strengthens the edge features through the efficient feature enhancement capabilities of the spatial attention layer and the channel attention layer. The channel attention layer can locate the information-rich channels in the features and enhance their weights. The spatial attention layer can focus on the edge positions in the edge features and enhance the edge features. It can enhance the segmentation network's attention to the edges and boundary positions, and improve the effectiveness and robustness of the network.

[0087] In addition, refer to Figure 3 In one embodiment, the channel attention layer includes a maximum pooling layer and an average pooling layer, and the channel attention layer in the attention module corresponding to the i-th edge detection network is called to map the i-th edge feature to obtain the channel attention feature, including but not limited to the following steps:

[0088] Step S310, for the channel attention layer in the attention module corresponding to the i-th edge detection network, call the maximum pooling layer in the channel attention layer to perform maximum pooling on the i-th edge feature to obtain the first channel attention sub-feature, and call the average pooling layer in the channel attention layer to perform average pooling on the i-th edge feature to obtain the second channel attention sub-feature.

[0089] Step S320, adding the first channel attention sub-feature and the second channel attention sub-feature to obtain a channel attention feature.

[0090] Among them, the channel attention feature can be expressed by the formula, the formula is as follows:

[0091] F e1 (x) = Conv(σ2(Conv(σ1(x))))

[0092] F e2 (x) = Conv(σ2(Conv(σ3(x))))

[0093] F map1 =Sigmoid(F e1 +F e2 )

[0094] Among them, F map1 is the channel attention feature, F e1 (x) is the first channel attention sub-feature, F e2 (x) is the second channel attention sub-feature, x is the i-th edge feature, and x∈R C×H×W , that is, x is a matrix with a height of H, a width of W, and a number of channels of C, σ1(·) is the adaptive average pooling, σ2(·) is the activation function Relu, σ3(·) is the adaptive maximum pooling, Conv(·) is the convolution operation, and Sigmoid(·) is the normalization operation.

[0095] Among them, referring to Figure 10, for the channel attention layer in the attention module corresponding to the i-th edge detection network, the adaptive maximum pooling operation can be to call the maximum pooling layer in the channel attention layer to perform maximum pooling on the i-th edge feature to obtain the first channel attention sub-feature, and the adaptive average pooling operation can be to call the maximum pooling layer in the channel attention layer to perform average pooling on the i-th edge feature to obtain the second channel attention sub-feature.

[0096] Among them, reference Fig.10 After obtaining the first channel attention sub-feature and the second channel attention sub-feature, the first channel attention sub-feature is input into the first convolution layer for convolution to complete feature extraction and obtain the third channel attention sub-feature. The third channel attention sub-feature is then input into the first activation layer for activation to obtain the fourth channel attention sub-feature. Finally, the fourth channel attention sub-feature is input into the second convolution layer for convolution to obtain the fifth channel attention sub-feature. Similarly, the second channel attention sub-feature is input into the third convolution layer for convolution to complete feature extraction and obtain the sixth channel attention sub-feature. The sixth channel attention sub-feature is then input into the second activation layer for activation to obtain the seventh channel attention sub-feature. Finally, the seventh channel attention sub-feature is input into the fourth convolution layer for convolution to obtain the eighth channel attention sub-feature. Finally, the fifth channel attention sub-feature and the eighth channel attention sub-feature are added and normalized to obtain the channel attention.

[0097] Among them, the first convolution layer, the second convolution layer, the third convolution layer and the fourth convolution layer can all be 3×3 convolution layers, the activation functions of the first activation layer and the second activation layer can be Relu, and the normalization operation can be completed by the Sigmoid function, which is not limited to the embodiments disclosed in this application.

[0098] Based on this, channel attention obtains features with different performances through the two branches of maximum pooling and average pooling, and adds the two features to complete feature fusion to obtain the channel attention map to enrich the information of edge features. It can locate the information-rich channels in the edge features and enhance their weights.

[0099] In addition, refer to Figure 4 In one embodiment, calling the spatial attention layer in the attention module corresponding to the ith edge detection network to map the ith edge feature to obtain the spatial attention feature includes but is not limited to the following steps:

[0100] Step S410, calling the spatial attention layer in the attention module corresponding to the i-th edge detection network, taking the maximum value of the i-th edge feature in the channel dimension to obtain the first spatial attention sub-feature, and taking the average value of the i-th edge feature in the channel dimension to obtain the second spatial attention sub-feature.

[0101] Step S420, determining the spatial attention feature based on the addition of the first spatial attention sub-feature and the second spatial attention sub-feature.

[0102] Among them, the maximum value of the i-th edge feature is taken in the channel dimension, specifically: for each pixel position of the i-th edge feature, the maximum value of all channel maps of the i-th edge feature at the same pixel position is determined, and a new feature map composed of the maximum values ​​corresponding to each pixel position is used as the first spatial attention sub-feature; similarly, for each pixel position of the i-th edge feature, the average value of all channel maps of the i-th edge feature at the same pixel position is determined, and a new feature map composed of the average values ​​corresponding to each pixel position is used as the second spatial attention sub-feature.

[0103] Among them, reference Fig.10 Channel operation 1 is to call the spatial attention layer in the attention module corresponding to the i-th edge detection network, and take the maximum value of the i-th edge feature in the channel dimension to obtain the first spatial attention sub-feature. Channel operation 2 is to take the average value of the i-th edge feature in the channel dimension to obtain the second spatial attention sub-feature. After adding the first spatial attention sub-feature and the second spatial attention sub-feature, they are input into the fifth convolutional layer for convolution to obtain the third spatial attention sub-feature. The third spatial attention sub-feature is normalized to obtain the spatial attention feature.

[0104] Among them, the fifth convolutional layer can be a 7×7 convolutional layer, which is not limited in the embodiments disclosed in this application.

[0105] Among them, the spatial attention feature can be expressed by the formula, the formula is as follows:

[0106] F map2 (x)=Sigmoid(Conv(β1(x)+β2(x)))

[0107] Among them, F map2 is the spatial attention feature, β1(·) is the function of taking the maximum value of the channel dimension, β2(·) is the function of taking the average value of the channel dimension, Conv(·) is the convolution operation, and Sigmoid is the normalization operation.

[0108] Based on this, the spatial attention layer obtains the first spatial attention sub-feature and the second spatial attention sub-feature through the maximum value and the average value processing on the channel dimension, and fuses the first spatial attention sub-feature and the second spatial attention sub-feature through convolution. Finally, the spatial attention feature is obtained through normalization operation. The spatial attention feature can strengthen the edge information of the first image feature, focus on the edge position in the feature, enhance the network's attention to the edge and boundary position, and improve the effectiveness and robustness of the network.

[0109] In addition, refer to Figure 5 In one embodiment, calling a decoding module in a decoding network to decode the first image feature to obtain an initial decoding feature includes but is not limited to the following steps:

[0110] Step S510: performing global feature extraction on the nth first image feature to obtain a second image feature.

[0111] Step S520, calling the decoding module corresponding to the nth local feature extraction network in the decoding network to decode the second image feature, and obtaining an initial decoding feature corresponding to the nth first image feature.

[0112] Step S530, based on the initial decoding feature corresponding to the nth first image feature, calling the decoding module corresponding to the n-1th local feature extraction network in the decoding network to decode the n-1th first image feature to obtain the initial decoding feature corresponding to the n-1th first image feature.

[0113] Step S540: Decoding is performed again based on the initial decoding feature corresponding to the (n-1)th first image feature, until the initial decoding features corresponding to each first image feature are obtained.

[0114] It should be noted that Transformer is a global structure network. Transformer is a global structure for learning the entire image. The global feature extraction of the nth first image feature can be completed through the Transformer network.

[0115] Among them, the decoding module corresponding to the nth local feature extraction network includes a sixth convolutional layer. The second image feature is input into the sixth convolutional layer for convolution, which can realize the decoding of the first image feature and obtain the initial decoding feature corresponding to the nth first image feature.

[0116] Among them, the input of the decoding module corresponding to the j-th local feature extraction network includes the j-th first image feature and the target attention feature output by the attention module corresponding to the j+1-th edge detection network, that is, the input of the decoding module corresponding to the j-th local feature extraction network includes the j-th first image feature and the target attention feature corresponding to the j+1-th edge feature. Therefore, based on the target attention feature corresponding to the j+1-th edge feature, the decoding module corresponding to the j-th local feature extraction network is called to decode the j-th first image feature to obtain the initial decoding feature corresponding to the j-th first image feature, where j is greater than or equal to 1, and j is less than or equal to 1. For n-2, for example, when n is 4, the input of the decoding module corresponding to the first local feature extraction network includes the first first image feature and the target attention feature output by the attention module corresponding to the second edge detection network, the input of the decoding module corresponding to the second local feature extraction network includes the second first image feature and the target attention feature output by the attention module corresponding to the third edge detection network, and the input of the decoding module corresponding to the third local feature extraction network includes the third first image feature and the initial decoding feature output by the fourth decoding module, and the input of the decoding module corresponding to the fourth local feature extraction network includes the second image feature.

[0117] Based on this, the global feature extraction is first performed on the nth first image feature belonging to the local feature, which can increase the global learning ability of the network. Then the decoding network starts to decode the second image feature until the initial decoding features output by each decoding module in the decoding network and the target attention features output by each attention module are finally obtained. This can enhance the boundary learning ability of the network model, enhance the segmentation network's attention to the boundary position, and improve the accuracy of medical image segmentation.

[0118] In addition, refer to Figure 6In one embodiment, the decoding module corresponding to the n-1th local feature extraction network includes an upsampling layer and an attention gate. Based on the initial decoding feature corresponding to the n-1th first image feature, the decoding module corresponding to the n-1th local feature extraction network in the decoding network is called to decode the n-1th first image feature to obtain the initial decoding feature corresponding to the n-1th first image feature, including but not limited to the following steps:

[0119] Step S610: call the upsampling layer to perform upsampling processing on the initial decoded feature corresponding to the nth first image feature to obtain a first sampling feature.

[0120] Step S620: Call the attention gate to perform attention processing on the first sampling feature based on the n-1th first image feature to obtain a first intermediate feature.

[0121] Step S630: concatenate the first sampling feature and the first intermediate feature to obtain an initial decoding feature corresponding to the n-1th first image feature.

[0122] Among them, the initial decoding feature corresponding to the nth first image feature is convolved through the upsampling layer, and bilinear interpolation, adaptive upsampling or nearest neighbor interpolation can be performed. Finally, the size of the above initial decoding feature is restored to be consistent with the size of the n-1th first image feature. The embodiment of the present application is not limited here.

[0123] Among them, the attention processing performed by the attention gate on the first sampling feature can be weighting the first sampling feature based on the n-1th first image feature to increase the weight of the part of the first sampling feature that is similar to the first image feature, and the embodiment of the present application is not limited here.

[0124] It should be noted that the structure of the decoding module corresponding to the j-th local feature extraction network is consistent with the structure of the decoding module corresponding to the n-1-th local feature extraction network, j is greater than or equal to 1, and j is less than or equal to n-2.

[0125] Among them, reference Fig.11 , the upper layer input can be the initial decoding feature corresponding to the nth first image feature, or the target attention feature corresponding to the ith edge feature. The upsampling convolution layer corresponds to the upsampling layer of the present application. A is the attention gate, and C is the splicing operation. For example, the initial decoding feature corresponding to the nth first image feature is convolved through the upsampling layer to obtain the first sampling feature, and the attention gate is called to perform attention processing on the first sampling feature based on the n-1th first image feature to obtain the first intermediate feature. The first sampling feature and the first intermediate feature are spliced ​​to obtain the initial decoding feature corresponding to the n-1th first image feature.

[0126] It should be noted that the reference Fig.13 The upsampling layer includes an upsampling operation layer, a seventh convolution layer, an activation layer using a Relu function, and a standardization operation layer. The input of the upsampling layer is processed in sequence by the upsampling operation layer, the seventh convolution layer, the activation layer using a Relu function, and the standardization operation layer. For example, the initial decoding feature corresponding to the nth first image feature is processed in sequence by the upsampling operation layer, the seventh convolution layer, the activation layer using a Relu function, and the standardization operation layer, and the final output is the first sampling feature.

[0127] Based on this, the upsampling layer is called to upsample the initial decoding feature corresponding to the nth first image feature to obtain the first sampling feature, which can restore the size of the above initial decoding feature to the same size as the n-1th first image feature, facilitating subsequent attention processing and splicing operations, and then the attention gate is called to perform attention processing on the first sampling feature based on the n-1th first image feature to obtain the first intermediate feature, which can enable the network to pay more attention to feature information of different sizes. Finally, the first sampling feature and the first intermediate feature are spliced ​​to obtain the initial decoding feature corresponding to the n-1th first image feature, realizing the fusion of multi-scale features, so that the initial decoding feature has rich multi-scale feature information, which is conducive to improving the accuracy and robustness of medical image segmentation.

[0128] In addition, refer to Figure 7 In one embodiment, the decoding network further includes convolution modules corresponding to the respective decoding modules, and performs image segmentation on the medical image based on the respective target attention features to obtain image segmentation results, including but not limited to the following steps:

[0129] Step S710, convolve the nth initial decoding feature to determine a first feature map.

[0130] Step S720, convolve each target attention feature respectively to determine the second feature map corresponding to each target attention feature.

[0131] Step S730, determining the category probability corresponding to each pixel in the medical image based on the fusion result of the first feature map and each second feature map.

[0132] Step S740, performing image segmentation on the medical image based on the category probability corresponding to each pixel to obtain an image segmentation result.

[0133] Among them, the dimensions of each second feature map and the first feature map are the same.

[0134] Among them, the category probability refers to the probability value that a pixel belongs to a classification category.

[0135] Among them, since the decoding network also includes convolution modules corresponding to each decoding module, there are a total of n convolution modules. The input of the nth convolution module is the nth initial decoding feature, and the input of the i-th convolution module is the i-th target attention feature.

[0136] Among them, the convolution module corresponding to each decoding module may include a double-layer convolution layer, which is not limited in the embodiment of the present application.

[0137] It should be noted that the first feature map and each second feature map correspond to a possible image segmentation result. The first feature map and each second feature map can be fused to obtain a fusion result of the first feature map and each second feature map. The fusion result can provide a more comprehensive basis for medical image segmentation, corresponding to a more accurate image segmentation result.

[0138] Among them, the first feature map and each second feature map can be added to obtain a fusion result, and the fusion result is normalized. Each pixel of the fusion result has a set of probability distributions corresponding to each category, and the category corresponding to the maximum probability in the probability distribution corresponding to each pixel is taken as the target category. Finally, the pixels with the same target category can be divided with the same label to obtain the image segmentation result. For example, the categories include forehead, left face, right face nose and chin. A pixel of the fusion result has a corresponding probability distribution: the corresponding probability of the forehead is 0.2, the corresponding probability of the left face is 0.3, the corresponding probability of the right face nose is 0.1, and the corresponding probability of the chin is 0.4. The chin has the largest probability. Therefore, the chin is the target category corresponding to the above pixel. If the mark corresponding to the chin is blue, the above pixel can be marked as blue, thereby realizing the classification of the above pixel into the chin category.

[0139] The fusion result can be normalized by a softmax function and an argmax function, which is not limited in the embodiment of the present application.

[0140] Based on this, the first feature map can provide more global image information of the medical image, and each second feature map contains edge information of the medical image at different scales and different weights. Therefore, each second feature map can help the network better identify edges and boundaries in the medical image, so that the fusion result of the first feature map and each second feature map has more global image edge information, so that the category probability corresponding to each pixel in the medical image is more accurate, thereby improving the accuracy of the image segmentation result.

[0141] It should be noted that the reference Figure 8, the input image is a medical image, the encoder is composed of various local feature extraction networks, the local feature extraction network is implemented based on the ResNetV2 module, the edge network is composed of various edge detection networks, the edge detection network is implemented based on the Res2Net module and the Sobel operator, the first feature extraction module of the edge detection network is implemented based on the Res2Net module, and the edge extraction module of the edge detection network is implemented based on the Sobel operator, Transformer is a global feature extraction network, and the decoder is a decoding network. First, the medical image is input into the encoder. The encoder includes 4 local feature extraction networks. Based on the 4 local feature extraction networks, cascade feature extraction is performed on the input medical image to obtain four first image features with successively reduced scales. The first three first image features are respectively input into the first three stages of the decoder, and the fourth first image feature is input into the global feature extraction network for global feature extraction to obtain the second image feature, and then the second image feature is input into the decoder. In the fourth stage of the encoder, at the same time, the medical image is input into the edge network, which includes three edge detection networks. First, based on the first feature extraction modules of each edge detection network, cascade feature extraction is performed on the medical image to obtain the intermediate feature map corresponding to each edge detection network, and then the corresponding intermediate feature map is input into the edge extraction module of each edge detection network for edge extraction to obtain the edge features corresponding to each edge detection network. The edge features corresponding to the three edge detection networks are respectively input into the first three stages of the decoder. The decoder decodes and pays attention to the input multiple first image features and multiple edge features to obtain the first feature map corresponding to the fourth first image feature, and the second feature maps corresponding to the first three first image features are obtained. At the same time, the three second feature maps also correspond to the three edge features one by one. Finally, the first feature map and each second feature map are fused to obtain a fusion result, and the medical image is segmented based on the fusion result to obtain an image segmentation result, i.e., the output image.

[0142] It should be noted that the reference Fig.14 During the training process, the edge detection network is also equipped with a predictor, whose input is the output of the edge extraction module, i.e., the edge feature. Fig.15 , the edge features are input into the predictor, and the edge features are sequentially subjected to convolution, activation, standardization, convolution, activation, standardization, and upsampling operations to finally obtain the output features, i.e., the prediction features; in addition, the loss function of the edge detection network is the Lovalz loss function, which can be expressed by the following formula:

[0143]

[0144] Among them, Loss Lovase is the model loss, y is the predicted feature, is the true label.

[0145] It is understandable that since the predictor can improve the integrity of the edge detection network and introduce the Lovaz loss function to determine the model loss, it can help the network learn and accelerate network convergence, thereby improving the network training efficiency and training effect.

[0146] It should be noted that the reference Fig. 9 , Fig. 9 An optional structural diagram of a decoder provided in an embodiment of the present application, the encoder has four stages, each stage has different inputs, wherein the decoding structure corresponds to the decoding module of the present application, and the edge attention structure corresponds to the attention module of the present application. For example, stage 4 of the encoder is provided with a sixth convolutional layer and a double-layer convolutional layer, the input of the sixth convolutional layer is the second image feature and the output is the initial decoding feature, and the double-layer convolutional layer of the above-mentioned initial encoding input stage 4 is convolved to obtain a first feature map; stages 1 to 3 of the encoder are all provided with a decoding module, an attention module and a double-layer convolutional layer, and the input of the decoding module of stage 3 is the third feature map. The input of the decoding module of stage 2 is the second first image feature and the initial decoding feature of stage 3, and the input of the decoding module of stage 1 is the first first image feature and the initial decoding feature of stage 2. In addition, the input of the attention module of stage 3 is the third edge feature and the initial decoding feature of stage 3, the input of the attention module of stage 2 is the second edge feature and the initial decoding feature of stage 2, and the input of the attention module of stage 1 is the first edge feature and the initial decoding feature of stage 1. The outputs of the double-layer convolutional layers of stage 3, stage 2, and stage 1 are all the second feature maps.

[0147] It is worth noting that during training, the model sub-loss can be determined based on the output of each double-layer convolutional layer, for example, L Seg1 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the decoding network in stage 1, L Seg2 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the decoding network in stage 2, L Seg3 is the model sub-loss determined based on the second feature map output by the double-layer convolutional layer of the decoding network in stage 3, L Seg4 It is the model sub-loss determined based on the first feature map output by the double-layer convolutional layer of the decoding network at stage 4.

[0148] Specifically, the model sub-loss can be expressed by the formula:

[0149]

[0150]

[0151] LSeg =αL Dice +βL CE

[0152] Among them, L CE is the cross entropy loss function, L Dice is the Dice loss function, L Seg is the segmentation loss function, L Seg Used to determine the model sub-loss, N is the total number of pixels in a medical image, is the predicted segmentation map, yi is the original segmentation map, α and β are coefficients, and α and β are set to 0.7 and 0.3 respectively.

[0153] It should be noted that the model loss of the entire network can be expressed by the following formula:

[0154] Loss = L Seg1 +L Seg2 +L Seg3 +L Seg4

[0155] Among them, Loss is the model loss of the entire network, L Seg1 , L Seg2 , L Seg3 and L Seg4 They are the model sub-losses corresponding to encoder stage 1, stage 2, stage 3, and stage 4 respectively.

[0156] It should be noted that the reference Fig.12 The double-layer convolution layer includes two convolution structures connected in series, and the convolution structure includes the eighth convolution layer, an activation layer using the Relu function, and a standardization operation layer.

[0157] In addition, refer to Fig.16 The present application also provides a medical image segmentation device 1600, comprising:

[0158] An acquisition module 1610 is used to acquire a medical image to be segmented;

[0159] A feature extraction module 1620 is used to call n local feature extraction networks to perform cascade feature extraction on the medical image to obtain the first image features extracted by each local feature extraction network, wherein n is a positive integer greater than 2;

[0160] The edge detection module 1630 is used to call n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each edge detection network;

[0161] The attention processing module 1640 is used to input each first image feature and each edge feature into the decoding network, call the decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, and call the attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and the i-th initial decoding feature is obtained by decoding the i-th first image feature and the i+1-th first image feature, where i is a positive integer less than n:

[0162] The segmentation module 1650 is used to perform image segmentation on the medical image based on each target attention feature to obtain an image segmentation result.

[0163] It can be understood that the specific implementation of the medical image segmentation device 1600 is basically the same as the specific implementation of the above-mentioned medical image segmentation method, and will not be repeated here.

[0164] In addition, refer to Fig.17 , Fig.17 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0165] The processor 1701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0166] The memory 1702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1702 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1702, and the processor 1701 is called to execute the medical image segmentation method of the embodiment of the present application.

[0167] Input / output interface 1703, used to implement information input and output;

[0168] Communication interface 1704, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0169] A bus 1705 that transmits information between the various components of the device (e.g., the processor 1701, the memory 1702, the input / output interface 1703, and the communication interface 1704);

[0170] The processor 1701 , the memory 1702 , the input / output interface 1703 and the communication interface 1704 are connected to each other in communication within the device via the bus 1705 .

[0171] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned medical image segmentation method.

[0172] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0173] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0174] It can be understood by those skilled in the art that Figures 1 to 17 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0175] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0177] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0178] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0182] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0183] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A medical image segmentation method, characterized in that: include: Acquire a medical image to be segmented; Calling n local feature extraction networks to perform cascade feature extraction on the medical image to obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; Calling n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each edge detection network; Input each of the first image features and each of the edge features into a decoding network, call a decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature, call an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each of the local feature extraction networks and an attention module corresponding to each of the edge detection networks, and i is a positive integer less than n; The medical image is segmented based on each of the target attention features to obtain an image segmentation result.

2. The medical image segmentation method according to claim 1, characterized in that: The attention module includes a channel attention layer and a spatial attention layer, and the calling of the attention module in the decoding network performs attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, including: Calling the channel attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a channel attention feature; Performing attention processing on the i-th initial decoding feature based on the channel attention feature to obtain an initial target attention feature; Calling the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a spatial attention feature; The initial target attention feature is subjected to attention processing based on the spatial attention feature to obtain a target attention feature.

3. The medical image segmentation method according to claim 2, characterized in that: The channel attention layer includes a maximum pooling layer and an average pooling layer, and the calling of the channel attention layer in the attention module corresponding to the i-th edge detection network maps the i-th edge feature to obtain a channel attention feature, including: For the channel attention layer in the attention module corresponding to the i-th edge detection network, call the maximum pooling layer in the channel attention layer to perform maximum pooling on the i-th edge feature to obtain a first channel attention sub-feature, and call the average pooling layer in the channel attention layer to perform average pooling on the i-th edge feature to obtain a second channel attention sub-feature; A channel attention feature is obtained by adding the first channel attention sub-feature and the second channel attention sub-feature.

4. The medical image segmentation method according to claim 2, characterized in that: The calling of the spatial attention layer in the attention module corresponding to the i-th edge detection network to map the i-th edge feature to obtain a spatial attention feature includes: Calling the spatial attention layer in the attention module corresponding to the ith edge detection network, taking the maximum value of the ith edge feature in the channel dimension to obtain a first spatial attention sub-feature, and taking the average value of the ith edge feature in the channel dimension to obtain a second spatial attention sub-feature; A spatial attention feature is determined based on the addition of the first spatial attention sub-feature and the second spatial attention sub-feature.

5. The medical image segmentation method according to claim 1, characterized in that: The calling of the decoding module in the decoding network to decode the first image feature to obtain an initial decoding feature includes: Performing global feature extraction on the nth first image feature to obtain a second image feature; Calling the decoding module corresponding to the nth local feature extraction network in the decoding network to decode the second image feature to obtain an initial decoding feature corresponding to the nth first image feature; Based on the initial decoding feature corresponding to the nth first image feature, calling the decoding module corresponding to the n-1th local feature extraction network in the decoding network to decode the n-1th first image feature, to obtain the initial decoding feature corresponding to the n-1th first image feature: Decoding is performed again based on the initial decoding feature corresponding to the (n-1)th first image feature until the initial decoding features corresponding to each of the first image features are obtained.

6. The medical image segmentation method according to claim 5, characterized in that: The decoding module corresponding to the n-1th local feature extraction network includes an upsampling layer and an attention gate, and based on the initial decoding feature corresponding to the nth first image feature, calling the decoding module corresponding to the n-1th local feature extraction network in the decoding network to decode the n-1th first image feature to obtain the initial decoding feature corresponding to the n-1th first image feature, including: Calling the upsampling layer to perform upsampling processing on the initial decoded feature corresponding to the nth first image feature to obtain a first sampling feature; Calling the attention gate to perform attention processing on the first sampling feature based on the n-1th first image feature to obtain a first intermediate feature; The first sampling feature and the first intermediate feature are concatenated to obtain an initial decoding feature corresponding to the n-1th first image feature.

7. The medical image segmentation method according to claim 5, characterized in that: The decoding network further includes convolution modules corresponding to the respective decoding modules, and the image segmentation of the medical image is performed based on the respective target attention features to obtain an image segmentation result, including: Convolving the nth initial decoding feature to determine a first feature map; Convolving each of the target attention features respectively to determine a second feature map corresponding to each of the target attention features, wherein the dimensions of each of the second feature maps and the first feature map are the same; Determine, based on the fusion result of the first feature map and each of the second feature maps, a category probability corresponding to each pixel in the medical image, wherein the category probability refers to a probability value that the pixel belongs to a classification category; The medical image is segmented based on the category probabilities corresponding to the pixels to obtain an image segmentation result.

8. A medical image segmentation device, characterized in that: include: An acquisition module, used for acquiring a medical image to be segmented; A feature extraction module, used for calling n local feature extraction networks to perform cascade feature extraction on the medical image to obtain first image features extracted by each of the local feature extraction networks, wherein n is a positive integer greater than 2; An edge detection module, used for calling n-1 edge detection networks to perform cascade edge detection on the medical image to obtain edge features detected by each edge detection network; an attention processing module, used for inputting each of the first image features and each of the edge features into a decoding network, calling a decoding module in the decoding network to decode the first image features to obtain an initial decoding feature, calling an attention module in the decoding network to perform attention processing on the i-th initial decoding feature based on the i-th edge feature to obtain a target attention feature, wherein the decoding network includes a decoding module corresponding to each local feature extraction network and an attention module corresponding to each edge detection network, and the i-th initial decoding feature is obtained by decoding the i-th first image feature and the i+1-th first image feature, where i is a positive integer less than n; A segmentation module is used to perform image segmentation on the medical image based on each of the target attention features to obtain an image segmentation result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the medical image segmentation method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the medical image segmentation method according to any one of claims 1 to 7 is implemented.

Images