Medical image segmentation method and device, equipment, medium and computer program product

Abstract

The invention relates to the field of image processing, and provides a medical image segmentation method, device and equipment, a medium and a computer program product, and the method comprises the steps: constructing a U-shaped segmentation network based on a historical medical image sample; taking the to-be-processed image as the input of a gradient enhanced convolution encoder to obtain a multi-layer feature map output by the gradient enhanced convolution encoder; inputting the multi-layer feature map into the object context cascade decoder, dividing an intermediate class prediction map generated in a target decoding stage into a plurality of objects in each decoding stage of the object context cascade decoder, and determining decoding features of the target decoding stage and object features of each object; segmenting the to-be-processed image to obtain an image segmentation result based on the enhancement feature output by the object context cascade decoder; the enhancement feature is determined based on the decoding feature and the object feature. According to the invention, accurate segmentation of the medical image is realized through the gradient enhancement convolution decoder and the object context cascade decoder.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a medical image segmentation method, device, equipment, medium and computer program product. BACKGROUND

[0002] Segmenting medical images plays an important role in disease judgment. Since medical image segmentation has high requirements for segmentation accuracy, existing segmentation schemes usually adopt a supervised learning mode of encoder-decoder structure, for example, a segmentation network based on a convolutional neural network (CNN). However, the above existing scheme still has many problems, for example, (1) the lack of transformer intrinsic bias requires a large amount of labeled data to make the training process converge; (2) existing research focuses on proposing effective encoder structures but ignores decoder structures; (3) the segmentation model based on the transformer has high complexity and is difficult to deploy in devices with limited computing resources. The problems existing in the above existing scheme result in that the segmentation result of the existing medical image segmentation scheme is not accurate enough. SUMMARY

[0003] The present application provides a medical image segmentation method, device, equipment, medium and computer program product to solve the technical problem that the existing medical image segmentation result is not accurate enough.

[0004] The present application provides a medical image segmentation method, comprising the following steps: constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; taking a to-be-processed image as the input of the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder; inputting the multi-layer feature map into the object context cascaded decoder, dividing an intermediate class prediction map generated by a target decoding stage into multiple objects, determining a decoding feature of the target decoding stage and an object feature of each object at each decoding stage of the object context cascaded decoder; segmenting the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result; the enhanced feature is determined based on the decoding feature and the object feature.

[0005] According to the medical image segmentation method provided by the present application, the U-shaped segmentation network based on historical medical image samples comprises: preprocessing historical medical images containing a segmentation target to obtain historical medical image samples; input the historical medical image sample as an input of a detail feature extraction module to obtain a detail feature map output by the detail feature extraction module; input the detail feature map into a multi-scale feature extraction module, and determine whether the gradient-enhanced convolutional encoder is trained based on a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module.

[0006] According to the medical image segmentation method provided by the application, the multi-scale feature extraction module comprises a channel splitting module and a global context extraction module; the input of the detail feature map into the multi-scale feature extraction module, and the determination of whether the gradient-enhanced convolutional encoder is trained based on the multi-resolution multi-layer feature representation output by the multi-scale feature extraction module comprise: The detail feature map is equally divided in the channel dimension through the channel splitting module to obtain a first split feature map and a second split feature map; The second split feature map is input into a global context extraction module to obtain a target feature map output by the global context extraction module; The multi-resolution multi-layer feature representation output by the multi-scale feature extraction module is determined based on the first split feature map and the target feature map.

[0007] According to the medical image segmentation method provided by the application, the U-shaped segmentation network is constructed based on historical medical image samples, and the method further comprises: determining a target class prediction map generated by each decoding stage based on the multi-resolution multi-layer feature representation; and dividing the target class prediction map into multiple objects; determining the feature of each object based on the class prediction map of the first decoding stage and the decoding feature of the second decoding stage; the first decoding stage and the second decoding stage are adjacent decoding stages; enhancing the same class pixels based on the decoding feature of each decoding stage and the feature of each object to determine the decoding result of each decoding stage; determining whether the object context cascaded decoder is trained based on the decoding result.

[0008] According to the medical image segmentation method provided by the application, the determination of the decoding result of each decoding stage based on the decoding feature of each decoding stage and the feature of each object comprises: determining a key matrix and a value matrix based on the feature of each object, and determining a query matrix based on the decoding feature of each decoding stage; determining the enhanced feature of each decoding stage based on the key matrix, the value matrix and the query matrix after feature compression; Determine the decoding result of each decoding stage based on the decoding feature and the enhanced feature of each decoding stage.

[0009] According to the medical image segmentation method provided by the application, the method comprises the following steps: The enhanced feature of each decoding stage is up-sampled to obtain the restored feature resolution of each decoding stage. In the case that the restored feature resolution is the same as the resolution of the multi-resolution multi-layer feature representation, it is determined that the object context cascading decoder is trained.

[0010] The application further provides a medical image segmentation device, comprising the following modules: The U-shaped segmentation network construction module is used to construct a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascading decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; The multi-layer feature map determination module is used to take the image to be processed as the input of the gradient-enhanced convolutional encoder to obtain the multi-layer feature map output by the gradient-enhanced convolutional encoder; The feature acquisition module is used to input the multi-layer feature map into the object context cascading decoder, divide the intermediate class prediction map generated by the target decoding stage into multiple objects in each decoding stage of the object context cascading decoder, determine the decoding feature of the target decoding stage and the object feature of each object; The image segmentation module is used to segment the image to be processed based on the enhanced feature output by the object context cascading decoder to obtain an image segmentation result; the enhanced feature is determined based on the decoding feature and the object feature.

[0011] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the medical image segmentation method according to any one of the above.

[0012] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the medical image segmentation method according to any one of the above.

[0013] The application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the medical image segmentation method according to any one of the above.

[0014] The medical image segmentation method, device, equipment, medium and computer program product provided by the application combine a gradient-enhanced convolution decoder and an object context cascaded decoder to form a U-shaped segmentation network; the gradient-enhanced convolution decoder is used to extract complete image feature representation, and the detailed information and the semantic information in each decoding layer complement each other, thereby reducing the amount of calculation and enhancing the learning ability of the network; the object context cascaded decoder uses the prior knowledge of the intermediate prediction map to enhance the same type of pixels, and continuously recovers the feature resolution through up-sampling, thereby realizing iterative adjustment in the decoding stage and accurate segmentation of the medical image. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0016] Figure 1 is one of the flowcharts of the medical image segmentation method provided by the application.

[0017] Figure 2 is the second flowchart of the medical image segmentation method provided by the application.

[0018] Figure 3 is the structural schematic diagram of the medical image segmentation device provided by the application.

[0019] Figure 4 is the structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the application more clear, the technical solutions in the application will be described clearly and completely in the following with reference to the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the application.

[0021] The medical image segmentation method, device, equipment, medium and computer program product of the application will be described below. Figures 1-4

[0022] Figure 1 is one of the flowcharts of the medical image segmentation method provided by the application, as shown in the figure, the method comprises the following: Figure 1 ​​Step 100, constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; Specifically, the construction of the U-shaped segmentation network provided by the present application specifically comprises the following implementation steps: 1. selecting a medical image dataset (i.e. historical medical image samples in the present embodiment) and pre-processing the batch data thereof, the pre-processing operations including adjusting the image window width and window level, data format conversion, random cropping, random flipping and random rotation; 2. constructing a gradient-enhanced convolutional encoder for learning the hierarchical feature representation of the input image; 3. constructing an object context cascaded decoder for generating complete and differentiated decoding features; 4. combining the gradient-enhanced convolutional decoder and the object context cascaded decoder to obtain the U-shaped segmentation network. The gradient-enhanced convolutional encoder comprises a detail feature extraction part (detail feature extraction module) and a multi-scale feature extraction part (multi-scale feature extraction module).

[0023] Step 200, taking the image to be processed as the input of the gradient-enhanced convolutional encoder to obtain the multi-layer feature map output by the gradient-enhanced convolutional encoder; Specifically, for the image to be processed input into the gradient-enhanced convolutional encoder, first, the detail feature map is obtained through the processing of the detail feature extraction module Conv-BN-ReLU Block. Conv-BN-ReLU Block is a basic module structure in convolutional neural networks, which is stacked by a convolution layer (Conv), a batch normalization layer (Batch Normalization, BN) and a ReLU (activation function) in sequence. Then, the detail feature map is taken as the input of the multi-scale feature extraction module, and the multi-layer feature representation is obtained through the processing of the multi-scale feature extraction module; the multi-scale feature extraction module Pooling-TransformerModule is a hybrid design combining pooling operation Pooling and Transformer architecture, which is used to enhance the multi-scale feature extraction capability.

[0024] Step 300, inputting the multi-layer feature map into the object context cascaded decoder, dividing the intermediate class prediction map generated by the target decoding stage into multiple objects in each decoding stage of the object context cascaded decoder, determining the decoding features of the target decoding stage and the object features of each object; Specifically, in the decoding stage, the multi-layer feature map is input into the object context cascaded decoder, and through multi-stage decoding of the object context cascaded decoder, an intermediate category prediction map is generated under the supervision of auxiliary loss, and then the whole image is divided into multiple objects based on the intermediate category prediction map. The category prediction map of the latter decoding stage is multiplied by the decoding feature of the former decoding stage to obtain the feature representation of each object.

[0025] Step 400: based on the enhanced feature output by the object context cascaded decoder, the image to be processed is segmented to obtain an image segmentation result; the enhanced feature is determined based on the decoding feature and the object feature.

[0026] Specifically, the decoding feature and the object feature representation of each stage are interacted through cross-attention, and the pixels of the same category are enhanced. The enhanced feature and the decoding feature are connected through a residual connection, spliced in the channel dimension, and then fused through a convolution kernel to output as the decoding feature of each stage. The above process is repeated, and the decoding feature is predicted and enhanced under the supervision of auxiliary loss until the decoding feature resolution and the original output are the same, and the result of segmenting the image to be processed is obtained.

[0027] In this embodiment, the gradient enhancement convolution decoder and the object context cascaded decoder are combined to form a U-shaped segmentation network; the gradient enhancement convolution decoder is used to extract complete image feature representation, and the detailed information and semantic information in each decoding layer complement each other, thereby reducing the amount of calculation and enhancing the learning ability of the network; the object context cascaded decoder enhances the same type of pixels using the prior knowledge of the intermediate prediction map, and continuously recovers the feature resolution through upsampling, realizes the iterative adjustment of the cascade in the decoding stage, and realizes the accurate segmentation of the medical image.

[0028] Figure 2 is a flowchart of the medical image segmentation method provided by the present application, as shown in Figure 2 The method can further include: Step 110: pre-processing a historical medical image containing a segmentation target to obtain a historical medical image sample; Step 120: inputting the historical medical image sample into a detail feature extraction module to obtain a detail feature map output by the detail feature extraction module; Step 130: inputting the detail feature map into a multi-scale feature extraction module, and determining whether the gradient enhancement convolution encoder is trained based on a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module.

[0029] Specifically, the historical medical image in the present application takes an abdominal CT scan image as an example, and the segmentation target in the historical medical image containing the segmentation target can be various abdominal organs (liver, kidney, spleen and pancreas). Wherein, the preprocessing of the historical medical image includes adjusting the image window width and window level, data format conversion, random cropping, random flipping and random rotation, etc., and the size of the historical medical image is unified to 128x128 (unit: pixel).

[0030] The construction process of the gradient enhanced convolutional encoder (used for learning the feature representation of the input image) is as follows: for the input image with a size of 128x128 (in this embodiment, the historical medical image sample), first, the detail feature is extracted through the detail feature extraction module CBR (Conv-BN-ReLU, convolutional layer-batch normalization layer-activation function stacking module), wherein Conv is a convolutional layer; BN is a batch normalization layer; and ReLU is an activation function. The calculation formula of the detail feature extraction is shown in formula 1, wherein, is the detail feature map.

[0031] ; (1) The detail feature map is input into the multi-scale feature extraction module (Pooling-Transformer Module, PTM) to obtain multi-layer feature representations (i.e. multi-layer feature maps in this embodiment) of different resolutions output by the multi-scale feature extraction module. Through the output multi-layer feature map, it is determined whether the training process of the gradient enhanced convolutional encoder is completed.

[0032] In this embodiment, the gradient enhanced convolutional encoder collects global information in the encoding stage and maximizes the gradient combination to improve the learning ability of the network, and mixes the detail features and semantic features at each encoding scale.

[0033] In one embodiment, the medical image segmentation method provided by the embodiment of the present application can further include: Step 131, splitting the detail feature map in the channel dimension by the channel splitting module to obtain a first split feature map and a second split feature map; Step 132, taking the second split feature map as the input of the global context extraction module to obtain the target feature map output by the global context extraction module; Step 133, determining the multi-resolution multi-layer feature representation output by the multi-scale feature extraction module based on the first split feature map and the target feature map.

[0034] Specifically, the multi-scale feature extraction module includes a channel-split module and a global context extraction module.

[0035] In the channel dimension, the channel split module splits the detail feature map into two parts, and the calculation formula is shown as formula 2, wherein, is the first split feature map, which retains part of the features in ; is the second split feature map; is split into .

[0036] ; (2) The global context extraction module takes the second split feature map as input, and models the global feature through PoolFormer (a lightweight visual model). PoolFormer retains the architecture of transformer, including multi-layer perceptron MLP, GN (a normalization technique used to replace BN in deep learning) and residual connection. The calculation of PoolFormer is shown as formula 3 to formula 5.

[0037] ; (3) ; (4) ; (5) ; (6) wherein, is the size of the pooling kernel; and is the multi-layer perceptron MLP; is the ReLU activation function; is the spatial position of the current point; is the spatial position of the sampling point; is the result containing global information after poolformer processing; is the concatenated with as the output result of the PTM block; is the final feature map of the th encoding stage. PTM captures long-range dependencies while maximizing gradient combinations, and the processing process is shown as formula 6, represents the operation of concatenation in the channel dimension; is the result of the stage feature extraction. In order to alleviate the deficiency of PoolFormer in extracting detail features, through residual connection with The addition is performed in the channel dimension. The above is repeated, and a plurality of feature maps of different resolutions are obtained through a pooling operation, and the gradient-enhanced convolutional encoder is trained.

[0038] The gradient-enhanced convolutional encoder provided in the application collects global information by embedding a pooling into a transformer general architecture, and relieves the huge calculation overhead of self-attention. The global information and the detail information complement each other through a residual connection, form global-local features in each scale, and finally form a hierarchical feature representation.

[0039] In one embodiment, the medical image segmentation method provided by the embodiment of the application can further include: Step 140, determining a target class prediction map generated by each decoding stage based on the multi-resolution multi-layer feature representation; and dividing the target class prediction map into a plurality of objects; Step 150, determining a feature of each object based on the class prediction map of the first decoding stage and the decoding feature of the second decoding stage; the first decoding stage and the second decoding stage are adjacent decoding stages; Step 160, enhancing the same class pixels based on the decoding feature of each decoding stage and the feature of each object, and determining a decoding result of each decoding stage; Step 170, determining whether the object context cascaded decoder is trained based on the decoding result.

[0040] The medical image segmentation method provided by the embodiment of the application can further include: Step 161, determining a key matrix and a value matrix based on the feature of each object, and determining a query matrix based on the decoding feature of each decoding stage; Step 162, determining an enhanced feature of each decoding stage based on the key matrix, the value matrix and the query matrix after feature compression; Step 163, determining a decoding result of each decoding stage based on the decoding feature and the enhanced feature of each decoding stage.

[0041] The medical image segmentation method provided by the embodiment of the application can further include: Step 171, performing up-sampling processing on the enhanced feature of each decoding stage to obtain a restored feature resolution of each decoding stage; Step 172, in the case that the restored feature resolution is the same as the resolution of the multi-resolution multi-layer feature representation, determining that the object context cascaded decoder is trained.

[0042] Specifically, the process of constructing the object context cascaded decoder (for continuously adjusting and refining the decoding feature) is as follows: The first stage of the object context cascaded decoder is supervised by the auxiliary loss The intermediate class prediction map generated in the decoding stage , which is divided into N objects, for each spatial point represents the pixel The probability that it belongs to each class, where , N≤Nc, Nc is the total number of classes in the (historical medical image sample) dataset.

[0043] The first stage of the object context cascaded decoder is supervised by the auxiliary loss The intermediate class prediction map generated in the decoding stage (i.e., the first decoding stage in this embodiment) is multiplied by the decoding feature of the second decoding stage (i.e., the second decoding stage in this embodiment) to obtain the feature representation of the N objects , where the decoding feature has the same resolution as the class prediction map , and is calculated as shown in equation 7.

[0044] ; (7) The decoding feature of the first stage interacts with the object feature representation (i.e., the feature of each object in this embodiment) through cross-attention to enhance the pixels of the same class. Wherein the object feature representation is calculated through a learnable transformation matrix matrix (i.e., the key matrix in this embodiment) and matrix (i.e., the value matrix in this embodiment); the decoding feature of the first decoding stage is calculated through a learnable transformation matrix matrix (i.e., the query matrix in this embodiment), as shown in equation 8. ; (8)

[0045] In order to reduce the computational overhead when interacting at a shallow level, the matrix, matrix and matrix are feature compressed, specifically by fixing the feature map resolution to 8x8 size through a pooling operation, as shown in equation 9, where is the max pooling.

[0046] ; (9) ​​​​Three matrices after the pooling operation, from multiple angles through multi-head cross-attention for feature enhancement, in the enhancement process, the introduction of learnable relative position encoding The difference information of different positions in the feature map is provided, and the calculation is as shown in formula 10, For The dimension of the matrix; The inner product can be prevented from being too large.

[0047] ; (10) Enhanced features And the decoding features Spliced through the residual connection in the channel dimension, and then fused through the 1*1 convolution kernel, output as the decoding feature of the first Stage (i.e. the decoding result in the embodiment), the calculation is as shown in formula 11.

[0048] ; (11) Repeat the above, under the supervision of auxiliary loss, the decoding feature is predicted and enhanced until the decoding feature resolution and the original input image resolution are the same, the training process of the object context cascading decoder is completed (in the decoding stage, the intermediate probability map containing prior knowledge is used to interact with the decoding feature to enhance the features of all pixels belonging to the same object. In the decoding process, the resolution of the feature map is constantly restored, and finally the accurate segmentation of the medical image is realized).

[0049] The object context cascading decoder proposed in the embodiment uses the output probability map of the intermediate decoding layer and the decoding feature to construct the object context feature, and enhances the features of all pixels belonging to the same object. In the enhancement process, feature compression and relative position encoding are introduced, which reduces the amount of cross-attention calculation while supplementing the position information. In the decoder stage, a cascading adjustment is formed to constantly refine the decoding feature.

[0050] The medical image segmentation device provided by the present application is described below, and the medical image segmentation device described below can be correspondingly referred to the medical image segmentation method described above.

[0051] Please refer to Figure 3 The present application also provides a medical image segmentation device, comprising: A U-shaped segmentation network construction module 301 is used to construct a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascading decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; The multi-layer feature map determination module 302 is configured to input the to-be-processed image into the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder. The feature acquisition module 303 is configured to input the multi-layer feature map into the object context cascaded decoder, divide an intermediate class prediction map generated by a target decoding stage into a plurality of objects, determine a decoding feature of the target decoding stage, and an object feature of each object. The image segmentation module 304 is configured to segment the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result, wherein the enhanced feature is determined based on the decoding feature and the object feature.

[0052] Optionally, the U-shaped segmentation network construction module comprises: The image preprocessing unit is configured to preprocess a historical medical image containing a segmentation target to obtain a historical medical image sample. The detail feature extraction unit is configured to input the historical medical image sample into a detail feature extraction module to obtain a detail feature map output by the detail feature extraction module. The multi-resolution multi-layer feature representation output unit is configured to input the detail feature map into a multi-scale feature extraction module, and determine whether the gradient-enhanced convolutional encoder is trained based on a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module.

[0053] Optionally, the multi-scale feature extraction module comprises a channel splitting module and a global context extraction module, and the multi-resolution multi-layer feature representation output unit comprises: The channel splitting unit is configured to split the detail feature map in a channel dimension by the channel splitting module to obtain a first split feature map and a second split feature map. The target feature map output unit is configured to input the second split feature map into the global context extraction module to obtain a target feature map output by the global context extraction module. The multi-layer feature extraction unit is configured to determine a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module based on the first split feature map and the target feature map.

[0054] Optionally, the U-shaped segmentation network construction module further comprises: The target class prediction map determination unit is configured to determine a target class prediction map generated by each decoding stage based on the multi-resolution multi-layer feature representation, and divide the target class prediction map into a plurality of objects. An object feature unit is configured to determine a feature of each object based on a class prediction map of a first decoding stage and a decoded feature of a second decoding stage, wherein the first decoding stage and the second decoding stage are adjacent decoding stages. A decoding result determination unit is configured to determine a decoding result of each decoding stage by enhancing same-class pixels based on the decoded feature of each decoding stage and the feature of each object. A first training confirmation unit is configured to determine whether the object context cascading decoder is trained based on the decoding result.

[0055] Optionally, the decoding result determination unit comprises: A query matrix determination unit is configured to determine a key matrix and a value matrix based on the feature of each object, and determine a query matrix based on the decoded feature of each decoding stage. A feature enhancement unit is configured to determine an enhanced feature of each decoding stage based on the key matrix, the value matrix and the query matrix after feature compression. A decoding result determination unit is configured to determine a decoding result of each decoding stage based on the decoded feature and the enhanced feature of each decoding stage.

[0056] Optionally, the determining whether the object context cascading decoder is trained based on the decoding result comprises: A restored feature resolution determination unit is configured to perform up-sampling processing on the enhanced feature of each decoding stage to obtain a restored feature resolution of each decoding stage. A second training confirmation unit is configured to determine that the object context cascading decoder is trained in a case where the restored feature resolution is the same as a resolution of the multi-resolution multi-layer feature representation.

[0057] Figure 4 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 4As shown, the electronic device can include a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 complete mutual communication through the communications bus 440. The processor 410 can invoke a logical instruction in the memory 430 to execute a medical image segmentation method, which includes: constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network includes a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder includes a detail feature extraction module and a multi-scale feature extraction module; taking a to-be-processed image as an input of the gradient-enhanced convolutional encoder, obtaining a multi-layer feature map output by the gradient-enhanced convolutional encoder; inputting the multi-layer feature map into the object context cascaded decoder, dividing an intermediate class prediction map generated by a target decoding stage into multiple objects in each decoding stage of the object context cascaded decoder, determining a decoding feature of the target decoding stage and an object feature of each object; segmenting the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result; and the enhanced feature is determined based on the decoding feature and the object feature.

[0058] In addition, the logical instructions in the memory 430 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0059] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being executable by a processor to enable a computer to perform the medical image segmentation method provided by any of the above methods, the method comprising: constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprising a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprising a detail feature extraction module and a multi-scale feature extraction module; inputting a to-be-processed image into the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder; inputting the multi-layer feature map into the object context cascaded decoder, dividing an intermediate class prediction map generated by a target decoding stage into multiple objects, determining a decoding feature of the target decoding stage and an object feature of each object at each decoding stage of the object context cascaded decoder; and segmenting the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result; the enhanced feature being determined based on the decoding feature and the object feature.

[0060] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement a medical image segmentation method provided by any of the above methods, the method comprising: constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprising a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprising a detail feature extraction module and a multi-scale feature extraction module; inputting a to-be-processed image into the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder; inputting the multi-layer feature map into the object context cascaded decoder, dividing an intermediate class prediction map generated by a target decoding stage into multiple objects, determining a decoding feature of the target decoding stage and an object feature of each object at each decoding stage of the object context cascaded decoder; and segmenting the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result; the enhanced feature being determined based on the decoding feature and the object feature.

[0061] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0062] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A medical image segmentation method characterized by, The method comprises the following steps: constructing a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; taking a to-be-processed image as the input of the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder; inputting the multi-layer feature map into the object context cascaded decoder, dividing an intermediate class prediction map generated by a target decoding stage into multiple objects, determining a decoding feature of the target decoding stage and an object feature of each object at each decoding stage of the object context cascaded decoder; segmenting the to-be-processed image based on an enhanced feature output by the object context cascaded decoder to obtain an image segmentation result; the enhanced feature is determined based on the decoding feature and the object feature.

2. The medical image segmentation method according to claim 1, characterized in that, The method of constructing the U-shaped segmentation network based on historical medical image samples comprises the following steps: preprocessing a historical medical image containing a segmentation target to obtain a historical medical image sample; taking the historical medical image sample as the input of the detail feature extraction module to obtain a detail feature map output by the detail feature extraction module; inputting the detail feature map into the multi-scale feature extraction module and determining whether the gradient-enhanced convolutional encoder is trained based on a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module.

3. The medical image segmentation method according to claim 2, characterized in that, The multi-scale feature extraction module comprises a channel splitting module and a global context extraction module; the step of inputting the detail feature map into the multi-scale feature extraction module and determining whether the gradient-enhanced convolutional encoder is trained based on a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module comprises the following steps: splitting the detail feature map in the channel dimension by the channel splitting module to obtain a first split feature map and a second split feature map; taking the second split feature map as the input of the global context extraction module to obtain a target feature map output by the global context extraction module; determining a multi-resolution multi-layer feature representation output by the multi-scale feature extraction module based on the first split feature map and the target feature map.

4. The medical image segmentation method according to claim 3, characterized in that, The method of constructing the U-shaped segmentation network based on historical medical image samples further comprises the following steps: determining a target class prediction map generated by each decoding stage based on the multi-resolution multi-layer feature representation; dividing the target class prediction map into multiple objects; determining a feature of each object based on a class prediction map of a first decoding stage and a decoding feature of a second decoding stage; the first decoding stage and the second decoding stage are adjacent decoding stages; enhancing the same type of pixels based on the decoding feature of each decoding stage and the feature of each object to determine a decoding result of each decoding stage; determining whether the object context cascaded decoder is trained based on the decoding result.

5. The medical image segmentation method according to claim 4, characterized in that, The step of enhancing the same type of pixels based on the decoding feature of each decoding stage and the feature of each object to determine a decoding result of each decoding stage comprises the following steps: determine a key matrix and a value matrix based on characteristics of each object, determine a query matrix based on decoding characteristics of each decoding stage; determine enhanced characteristics of each decoding stage based on the key matrix, the value matrix and the query matrix after feature compression; determine a decoding result of each decoding stage based on the decoding characteristics and the enhanced characteristics of each decoding stage.

6. The medical image segmentation method according to claim 5, characterized in that, The determining whether the object context cascaded decoder is trained based on the decoding result comprises: performing up-sampling processing on the enhanced characteristics of each decoding stage to obtain a restored feature resolution of each decoding stage; In a case where the restored feature resolution is the same as a resolution of the multi-resolution multi-layer feature representation, it is determined that the object context cascaded decoder is trained.

7. A medical image segmentation apparatus characterized by comprising: comprise: a U-shaped segmentation network construction module configured to construct a U-shaped segmentation network based on historical medical image samples; the U-shaped segmentation network comprises a gradient-enhanced convolutional encoder and an object context cascaded decoder; the gradient-enhanced convolutional encoder comprises a detail feature extraction module and a multi-scale feature extraction module; a multi-layer feature map determination module configured to take a to-be-processed image as an input of the gradient-enhanced convolutional encoder to obtain a multi-layer feature map output by the gradient-enhanced convolutional encoder; a feature acquisition module configured to input the multi-layer feature map into the object context cascaded decoder, divide an intermediate class prediction map generated by a target decoding stage into a plurality of objects, determine decoding characteristics of the target decoding stage and object characteristics of each object at each decoding stage of the object context cascaded decoder; an image segmentation module configured to segment the to-be-processed image based on enhanced characteristics output by the object context cascaded decoder to obtain an image segmentation result; the enhanced characteristics are determined based on the decoding characteristics and the object characteristics.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the medical image segmentation method according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the medical image segmentation method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the medical image segmentation method according to any one of claims 1 to 6.

Images