Medical image segmentation method and device, electronic equipment and storage medium

Abstract

The application provides a medical image segmentation method and device, electronic equipment and storage medium, and belongs to the technical field of medical image processing. The source domain medical image and the target domain medical image are obtained, the source domain medical image and the target domain medical image are subjected to style conversion, the style migration feature map of the target domain medical image is obtained, the high-frequency feature extraction is performed on the style migration feature map, the first high-frequency feature map is obtained, the first high-frequency feature map is subjected to feature extraction through the preset first UNet network, the first feature map and the second feature map are obtained, the feature map splicing is performed on the first feature map and the second feature map, the third feature map is obtained, the image segmentation processing is performed on the third feature map through the preset second UNet network, and the image segmentation result of the target domain medical image is obtained, so that the accuracy of medical image segmentation can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a medical image segmentation method and device, an electronic device and a storage medium. BACKGROUND

[0002] Due to inconsistent shooting environments, various disease conditions of patients, and other reasons, a medical image usually contains a large amount of noise. Moreover, the features of medical images in different data sets are quite different, and the offset in the data domain will affect the segmentation result of the medical image. Noise and data domain offset will both affect the accuracy of image segmentation, and thus the medical image cannot be effectively segmented. Therefore, how to improve the accuracy of medical image segmentation has become a technical problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a medical image segmentation method, a medical image segmentation device, an electronic device and a storage medium, which aims to improve the accuracy of medical image segmentation.

[0004] To achieve the above-mentioned purpose, a first aspect of the embodiments of the present application provides a medical image segmentation method, which comprises the following steps:

[0005] obtaining a source domain medical image and a target domain medical image;

[0006] performing style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image;

[0007] performing high-frequency feature extraction on the style transfer feature map to obtain a first high-frequency feature map;

[0008] performing feature extraction on the first high-frequency feature map through a preset first UNet network to obtain a first feature map and a second feature map;

[0009] performing feature map splicing on the first feature map and the second feature map to obtain a third feature map;

[0010] performing image segmentation processing on the third feature map through a preset second UNet network to obtain an image segmentation result of the target domain medical image.

[0011] In some embodiments, the performing style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image comprises:

[0012] performing image domain transformation on the source domain medical image to obtain a first image frequency domain graph of the source domain medical image, and performing image domain transformation on the target domain medical image to obtain a second image frequency domain graph of the target domain medical image; wherein the first image frequency domain graph comprises a first amplitude graph and a first phase graph, and the second image frequency domain graph comprises a second amplitude graph;

[0013] performing image fusion on the first amplitude graph and the second amplitude graph to obtain a target amplitude graph;

[0014] performing frequency domain graph construction according to the target amplitude graph and the first phase graph to obtain a target frequency domain graph;

[0015] performing image domain restoration on the target frequency domain graph to obtain the style transfer feature graph.

[0016] In some embodiments, the performing image domain transformation on the source domain medical image to obtain a first image frequency domain graph of the source domain medical image, and performing image domain transformation on the target domain medical image to obtain a second image frequency domain graph of the target domain medical image comprises:

[0017] performing Fourier transform on the source domain medical image to obtain the first image frequency domain graph, and performing Fourier transform on the target domain medical image to obtain the second image frequency domain graph.

[0018] In some embodiments, the performing high-frequency feature extraction on the style transfer feature graph to obtain a first high-frequency feature graph comprises:

[0019] performing Gaussian filtering processing on the style transfer feature graph to obtain a filtered feature graph;

[0020] performing image subtraction on the style transfer feature graph and the filtered feature graph to obtain the first high-frequency feature graph.

[0021] In some embodiments, the first UNet network comprises a first double convolutional layer and a second double convolutional layer, and the first UNet network is preset to perform feature extraction on the first high-frequency feature graph to obtain a first feature graph and a second feature graph, comprising:

[0022] the first double convolutional layer is used to perform feature extraction on the first high-frequency feature graph to obtain the first feature graph;

[0023] the second double convolutional layer is used to perform feature extraction on the first feature graph to obtain the second feature graph.

[0024] In some embodiments, the first double convolutional layer includes a first convolutional layer, a first batch normalization layer, a first activation layer, a second convolutional layer, a second batch normalization layer, and a second activation layer, and the feature extraction of the first high-frequency feature map through the first double convolutional layer includes:

[0025] The first high-frequency feature map is convoluted through the first convolutional layer to obtain a first convolutional feature map, the first convolutional feature map is batch normalized through the first batch normalization layer to obtain a second convolutional feature map, and the second convolutional feature map is activated through the first activation layer to obtain an activated feature map.

[0026] The activated feature map is convoluted through the second convolutional layer to obtain a third convolutional feature map, the third convolutional feature map is batch normalized through the second batch normalization layer to obtain a fourth convolutional feature map, and the fourth convolutional feature map is activated through the second activation layer to obtain the first feature map.

[0027] In some embodiments, after the image segmentation of the third feature map through the preset second UNet network is performed to obtain the image segmentation result of the target domain medical image, the medical image segmentation method further includes:

[0028] performing high-frequency feature extraction on the source domain medical image to obtain a second high-frequency feature map; wherein the source domain medical image has an image segmentation label;

[0029] performing first loss calculation on the second feature map and the second high-frequency feature map to obtain high-frequency loss data;

[0030] performing second loss calculation on the image segmentation label and the image segmentation result to obtain segmentation loss data;

[0031] obtaining target loss data according to the high-frequency loss data and the segmentation loss data;

[0032] updating network parameters of the first UNet network and the second UNet network according to the target loss data.

[0033] To achieve the above object, a second aspect of the embodiment of the present application proposes a medical image segmentation device, which includes:

[0034] an acquisition module configured to acquire a source domain medical image and a target domain medical image;

[0035] a style conversion module configured to perform style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image;

[0036] a first feature extraction module configured to perform high-frequency feature extraction on the style transfer feature map to obtain a first high-frequency feature map;

[0037] a second feature extraction module configured to perform feature extraction on the first high-frequency feature map by using a preset first UNet network to obtain a first feature map and a second feature map;

[0038] a feature splicing module configured to perform feature map splicing on the first feature map and the second feature map to obtain a third feature map;

[0039] a medical image segmentation module configured to perform image segmentation processing on the third feature map by using a preset second UNet network to obtain an image segmentation result of the target domain medical image.

[0040] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0041] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0042] The medical image segmentation method, medical image segmentation device, electronic device and computer readable storage medium provided by the present application can obtain a source domain medical image and a target domain medical image, perform style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image, and through image style conversion, the target domain medical image can retain the original image content while obtaining the image style of the source domain medical image, thereby reducing the feature difference between the source domain medical image and the target domain medical image. Further, high-frequency feature extraction is performed on the style transfer feature map to obtain a first high-frequency feature map, through high-frequency feature extraction, the noise in the medical image can be reduced, and the feature difference between the source domain medical image content and the target domain medical image content in the style transfer feature map can be further reduced, thereby reducing the unique style features of the data domain. Furthermore, feature extraction is performed on the first high-frequency feature map by using a preset first UNet network to obtain a first feature map and a second feature map, in order to avoid information loss, feature map splicing is performed on the first feature map and the second feature map to obtain a third feature map. Finally, image segmentation processing is performed on the third feature map by using a preset second UNet network to obtain an image segmentation result of the target domain medical image, which can improve the accuracy of medical image segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flowchart of the medical image segmentation method provided by an embodiment of the present application;

[0044] Figure 2 is Figure 1 is a flowchart of step S120 in

[0045] Figure 3 is Figure 1 is a flowchart of step S130 in

[0046] Figure 4 is Figure 1 is a flowchart of step S140 in

[0047] Figure 5 is Figure 4 is a flowchart of step S410 in

[0048] Figure 6 is another flowchart of the medical image segmentation method provided by an embodiment of the present application;

[0049] Figure 7 is a structural schematic diagram of the medical image segmentation device provided by an embodiment of the present application;

[0050] Figure 8 is a hardware structural schematic diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0052] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0054] Medical image processing is one of the indispensable processes in automated medical systems, and medical image segmentation is one of the most important sub-tasks in medical image processing tasks. Medical images include human tissue and organ images obtained through various imaging methods such as X-ray, CT, MRI, fundus color photography, and are an important basis for medical institutions to carry out disease diagnosis, surgical planning, prognosis evaluation, and follow-up actions. Medical images contain a wealth of information, and this information is mostly present in the morphological structure of some regions of the image, so medical image segmentation is of great significance.

[0055] Image segmentation refers to distinguishing pixels in regions of the image that have special semantics. Existing medical image segmentation methods include manual segmentation and automated segmentation, but manual segmentation is not only time-consuming and labor-intensive, but also cannot guarantee the accuracy of the segmentation results. Compared with experimental data, real medical images often contain a lot of noise due to inconsistent shooting environments, diverse patient disease conditions, and other reasons. Moreover, the features of medical images in different data sets differ greatly, and this domain shift affects the segmentation performance of automated segmentation methods and the segmentation results of medical images. Noise and domain shift both affect the accuracy of image segmentation, making it difficult to effectively segment medical images, so how to improve the accuracy of medical image segmentation has become a technical problem to be solved.

[0056] Based on this, the embodiments of the present application provide a medical image segmentation method, a medical image segmentation device, an electronic device and a computer readable storage medium, aiming to improve the accuracy of medical image segmentation.

[0057] The medical image segmentation method, medical image segmentation device, electronic device and computer readable storage medium provided by the embodiments of the present application are specifically explained by the following embodiments, first, the medical image segmentation method in the embodiments of the present application is described.

[0058] The medical image segmentation method provided by the embodiments of the present application relates to the technical field of medical image processing. The medical image segmentation method provided by the embodiments of the present application can be applied in a terminal, can be applied in a server end, and can also be software running in a terminal or a server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as a standalone physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms; and the software can be an application that implements the medical image segmentation method, but is not limited to the above forms.

[0059] The application is operable in a variety of general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0060] Figure 1 is an optional flowchart of a medical image segmentation method provided by an embodiment of the application, Figure 1 The method in the method can include but is not limited to including steps S110 to S160.

[0061] Step S110, acquiring a source domain medical image and a target domain medical image;

[0062] Step S120, performing style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image;

[0063] Step S130, performing high-frequency feature extraction on the style transfer feature map to obtain a first high-frequency feature map;

[0064] Step S140, performing feature extraction on the first high-frequency feature map through a preset first UNet network to obtain a first feature map and a second feature map;

[0065] Step S150, performing feature map splicing on the first feature map and the second feature map to obtain a third feature map;

[0066] Step S160, performing image segmentation processing on the third feature map through a preset second UNet network to obtain an image segmentation result of the target domain medical image.

[0067] The steps S110 to S160 shown in the embodiments of the present application are as follows: the source domain medical image and the target domain medical image are obtained, the source domain medical image and the target domain medical image are style converted, the style transfer feature map of the target domain medical image is obtained, the image style conversion can make the target domain medical image retain the original image content while obtaining the image style of the source domain medical image, and the feature difference between the source domain medical image and the target domain medical image is reduced. Further, the high-frequency feature of the style transfer feature map is extracted to obtain the first high-frequency feature map. Through high-frequency feature extraction, the noise in the medical image can be reduced, and the feature difference between the source domain medical image content and the target domain medical image content in the style transfer feature map is further reduced, and the unique style feature of the data domain is reduced. Further, the first high-frequency feature map is extracted by the preset first UNet network to obtain the first feature map and the second feature map. In order to avoid information loss, the first feature map and the second feature map are spliced to obtain the third feature map. Finally, the third feature map is processed by the preset second UNet network to obtain the image segmentation result of the target domain medical image, which can improve the accuracy of medical image segmentation.

[0068] In step S110 of some embodiments, in the training process of the medical image segmentation model, a source domain medical image is obtained from a source domain data set (training set) every round, and a plurality of target domain medical images are obtained from a target domain data set. Each source domain medical image of the source domain data set has an image segmentation label, each target domain medical image of the target domain data set does not have an image segmentation label, and the image style is diverse. Specifically, a source domain medical image x s is randomly obtained from the training set every round, and K target domain medical images K is an integer greater than or equal to 1.

[0069] Please refer to Figure 2 In some embodiments, step S120 can include but is not limited to steps S210 to S240:

[0070] Step S210, performing image domain transformation on the source domain medical image to obtain a first image frequency domain map of the source domain medical image, and performing image domain transformation on the target domain medical image to obtain a second image frequency domain map of the target domain medical image; wherein the first image frequency domain map includes a first amplitude map and a first phase map, and the second image frequency domain map includes a second amplitude map;

[0071] Step S220, performing image fusion on the first amplitude map and the second amplitude map to obtain a target amplitude map;

[0072] Step S230, constructing a target frequency domain map according to the target amplitude map and the first phase map.

[0073] In step S240, the target frequency domain graph is obtained by image domain restoration.

[0074] In step S210 of some embodiments, the source domain medical image x s and each target domain medical image is converted to obtain the style transfer feature map of each target domain medical image where 1≤i≤K i∈Z. Specifically, the source domain medical image and the K target domain medical images are respectively transformed from the spatial domain to the frequency domain. The source domain medical image is subjected to a fast Fourier transform to obtain a first image frequency domain graph, and the target domain medical image is subjected to a fast Fourier transform to obtain a second image frequency domain graph. The first image frequency domain graph is split to obtain a first amplitude graph and a first phase graph, and the second image frequency domain graph is split to obtain a second amplitude graph and a second phase graph.

[0075] In step S220 of some embodiments, the first amplitude graph is represented as f s A , the first phase graph is represented as f s P , the second amplitude graph is represented as f t A , and the second phase graph is represented as f t P According to a certain proportion, the first amplitude graph f s A and the second amplitude graph f t A are added to obtain a target amplitude graph lf s A +(1-l)f t A , where l is an adjustable parameter between [0, 1].

[0076] In step S230 of some embodiments, the target amplitude graph lf s A +(1-l)f t A and the first phase graph f s P are constructed to obtain a target frequency domain graph, which is (lf s A +(1-l)f t A )exp(jf s P ).

[0077] It should be noted that, lf s A exp(jf s P ) is the low-frequency information on the frequency domain of the source domain medical image, (1-l)f t A exp(jf s P ) is the low-frequency information on the frequency domain of the target domain medical image, and the directional conversion of the image style is completed by fusing the low-frequency information on the frequency domain of the two medical images.

[0078] In step S240 of some embodiments, the target frequency domain image is restored to the image domain to restore the target frequency domain image from the frequency domain to the spatial domain to obtain the style migration feature map. Specifically, the inverse fast Fourier transform is performed on the target frequency domain image to obtain the style migration feature map x m The calculation method of the style migration feature map x

[0079] x m = F -1 [(lf s A +(1-l)f t A )exp(jf s P )] Formula (1)

[0080] Wherein, exp(jf s P ) is

[0081] The above steps S210 to S240 can reduce the feature difference between images, reduce the unique style of the data domain, and avoid the influence of the deviation on the data domain on the correctness of the segmentation result by performing image style conversion on the source domain medical image and the target domain medical image. The unique style of the data domain refers to the similar features in the same data set and the large differences between different data sets.

[0082] Please refer to Figure 3 In some embodiments, step S130 can include but is not limited to steps S310 to S320:

[0083] Step S310, Gaussian filtering is performed on the style migration feature map to obtain a filtered feature map;

[0084] Step S320, image difference is performed on the style migration feature map and the filtered feature map to obtain a first high-frequency feature map.

[0085] In step S310 of some embodiments, a Gaussian kernel size r and a standard deviation σ are determined, a Gaussian filter G(r, σ) is constructed according to the Gaussian kernel size and the standard deviation, and the style transfer feature map x m Gaussian filtering, that is, Gaussian blur, is performed on the image, and the image after Gaussian filtering is taken as a filtered feature map, where the Gaussian kernel size is set to 23 and the standard deviation is set to 20. It should be noted that the Gaussian filter is a low-pass filter, and therefore the filtered feature map is low-frequency information.

[0086] In step S320 of some embodiments, the style transfer feature map and the filtered feature map are traversed, the pixel value of a pixel point in the style transfer feature map is subtracted from the pixel value of a pixel point at the same position in the filtered feature map, and after all the pixel points are subtracted, a first high-frequency feature map is obtained. Since the difference between the style transfer feature map and the filtered feature map is not large, in order to increase the difference between the two, the first high-frequency feature map is enhanced according to a preset weight coefficient, and an enhanced first high-frequency feature map is obtained. The calculation method of the enhanced first high-frequency feature map H(x m ) is shown in formula (2).

[0087] H(x m )=α(x m -x m *G(r,σ)) Formula (2)

[0088] Wherein, α is a weight coefficient, which can be set to 4.

[0089] It should be noted that the image frequency of the background region of the style transfer image is low, and the influence of Gaussian blur on the background region is smaller than that on the region of interest, and therefore the difference between the filtered feature map and the style transfer feature map of the background region is smaller.

[0090] Through the above steps S310 to S320, the style information on the background region of the style transfer feature map can be reduced. By reducing the style information, the medical image features between different data sets can be made closer, and the segmentation performance of the cross-data-domain medical image can be improved.

[0091] Please refer to Figure 4 , in some embodiments, the first UNet network includes a first double convolutional layer and a second double convolutional layer, and step S140 can include but is not limited to steps S410 to S420:

[0092] In step S410, the first high-frequency feature map is feature-extracted through the first double convolutional layer to obtain a first feature map;

[0093] In step S420, the first feature map is feature-extracted through the second double convolutional layer to obtain a second feature map.

[0094] In step S410 of some embodiments, the first dual convolution layer is the first dual convolution layer (input layer) of the first UNet network, and the first feature map is a feature vector output by the first dual convolution layer.

[0095] In step S420 of some embodiments, the second dual convolution layer includes dual convolution layers of the left branch and the right branch of the first UNet network, the first feature map is subjected to feature extraction through the second dual convolution layer to obtain an intermediate feature map, and the intermediate feature map is subjected to convolution processing through a 1x1 convolution layer of an output layer of the first UNet network to obtain a second feature map.

[0096] In steps S410 to S420 described above, low-frequency information also exists in the first high-frequency feature map, and the low-frequency information gradually disappears as the network depth of the first UNet network increases and cannot be transmitted to the next layer of the network. By performing feature extraction on the first high-frequency feature map through the first UNet network, the style information strongly related to the data domain, such as image brightness, subject color, and blur degree, can be further reduced.

[0097] Please refer to Figure 5 In some embodiments, the first dual convolution layer includes a first convolution layer, a first batch normalization layer, a first activation layer, a second convolution layer, a second batch normalization layer, and a second activation layer, and step S410 can include but is not limited to steps S510 to S520:

[0098] In step S510, the first high-frequency feature map is subjected to convolution processing through the first convolution layer to obtain a first convolution feature map, the first convolution feature map is subjected to batch normalization processing through the first batch normalization layer to obtain a second convolution feature map, and the second convolution feature map is subjected to activation processing through the first activation layer to obtain an activated feature map.

[0099] In step S520, the activated feature map is subjected to convolution processing through the second convolution layer to obtain a third convolution feature map, the third convolution feature map is subjected to batch normalization processing through the second batch normalization layer to obtain a fourth convolution feature map, and the fourth convolution feature map is subjected to activation processing through the second activation layer to obtain the first feature map.

[0100] In step S510 of some embodiments, the first convolutional layer is a 3x3 convolutional layer, the first batch normalization layer is a BN (Batch Normalization) layer, and the first activation layer is a ReLU activation layer. Specifically, the first high-frequency feature map is convoluted by the 3x3 convolutional layer to obtain a first convolutional feature map, the first convolutional feature map is batch normalized by the BN layer to obtain a second convolutional feature map, and the second convolutional feature map is activated by the ReLU activation layer to obtain an activated feature map. Through batch normalization processing, the second convolutional feature map is subjected to a normal distribution with a mean of 0 and a variance of 1, avoiding variable distribution deviation, thereby avoiding gradient disappearance and gradient explosion, accelerating the training speed of the model while improving the generalization ability of the model.

[0101] In step S520 of some embodiments, the second convolutional layer is a 3x3 convolutional layer, the second batch normalization layer is a BN (Batch Normalization) layer, and the second activation layer is a ReLU activation layer. Specifically, the activated feature map is convoluted by the 3x3 convolutional layer to obtain a third convolutional feature map, the third convolutional feature map is batch normalized by the BN layer to obtain a fourth convolutional feature map, and the fourth convolutional feature map is activated by the ReLU activation layer to obtain a first feature map.

[0102] The steps S510 to S520 described above extract features from the first high-frequency feature map to obtain the intermediate output of the first UNet network, which facilitates subsequent image segmentation based on the intermediate output to improve the accuracy of image segmentation.

[0103] In step S150 of some embodiments, to avoid information loss, the first feature map and the second feature map are spliced to obtain a third feature map. For example, the first feature map is denoted as A1, the second feature map is denoted as A2, and the third feature map is denoted as A3, then A3 = [A1, A2].

[0104] In step S160 of some embodiments, the image style conversion, high-frequency feature extraction, and the first UNet network are combined into a self-supervised mechanism, and the third feature map is segmented by the second UNet network, which completes the segmentation of the target domain medical image in a self-supervised manner without using segmentation labels, while enhancing the robustness of the second UNet network to noise and the generalization of the second UNet network to data domains.

[0105] Referring to Figure 6 In some embodiments, after step S160, the medical image segmentation method can include, but is not limited to, steps S610 to S650:

[0106] Step S610, high-frequency feature extraction is performed on the source domain medical image to obtain a second high-frequency feature map; wherein the source domain medical image has an image segmentation label;

[0107] Step S620, first loss calculation is performed on the second feature map and the second high-frequency feature map to obtain high-frequency loss data;

[0108] Step S630, second loss calculation is performed on the image segmentation label and the image segmentation result to obtain segmentation loss data;

[0109] Step S640, target loss data is obtained according to the high-frequency loss data and the segmentation loss data;

[0110] Step S650, the network parameters of the first UNet network and the second UNet network are updated according to the target loss data.

[0111] In step S610 of some embodiments, the source domain medical image x s is subjected to Gaussian filtering through the Gaussian filter G constructed in step S310 to obtain a Gaussian blurred image, and the source domain medical image x s is subtracted from the Gaussian blurred image to obtain a second high-frequency feature map H(x s ).

[0112] In step S620 of some embodiments, first loss calculation is performed on the second feature map and the second high-frequency feature map H(x s ) to obtain high-frequency loss data, wherein the calculation method of the high-frequency loss data is shown in formula (3).

[0113]

[0114] wherein L high is the high-frequency loss data, K is the number of target domain medical images, and H seg is the kth first high-frequency feature map input into the first UNet network.

[0115] In step S630 of some embodiments, second loss calculation is performed on the image segmentation label of the source domain medical image and the image segmentation result of the target domain medical image to obtain segmentation loss data, wherein the calculation method of the segmentation loss data is shown in formula (4).

[0116]

[0117] wherein L seg is the segmentation loss data, K is the number of target domain medical images, y s is the image segmentation label of the source domain medical image x s , is the image segmentation result of the kth target domain medical image.

[0118] In step S640 of some embodiments, the high-frequency loss data and the segmentation loss data are added to obtain target loss data. The calculation method of the target loss data is shown in formula (5).

[0119] L total = L high + L seg Formula (5)

[0120] wherein, L total is the target loss data, that is, the target loss value.

[0121] In step S650 of some embodiments, the network parameters of the first UNet network and the second UNet network are updated according to the target loss data, and the training is stopped when the target loss data reaches the minimum, to obtain the trained first UNet network and the second UNet network. The target medical image is obtained, the target medical image is input into the trained first UNet network to reduce the style information strongly related to the data domain, to obtain the first feature map and the second feature map, the first feature map and the second feature map are spliced to obtain the third feature map, and the third feature map is input into the trained second UNet network for image segmentation to obtain the image segmentation result. Through the two cascaded UNet networks, the accuracy of medical image segmentation is improved.

[0122] Through the above steps S610 to S650, the parameters of the first UNet network and the second UNet network can be updated to obtain the trained first UNet network and the second UNet network. Based on the trained first UNet network and the second UNet network, the medical image is segmented to improve the accuracy of medical image segmentation.

[0123] The medical image segmentation method of the present application is described below with one specific embodiment.

[0124] The first fundus image and the K second fundus images are obtained, the first fundus image has a blood vessel segmentation label, the K second fundus images all do not have the blood vessel segmentation label and the image style is various, the first fundus image and each second fundus image are converted in image style, K style transfer images are obtained, the high-frequency component extraction is carried out to each style transfer image, K first high-frequency feature maps are obtained, each first high-frequency feature map is input to the first UNet network to carry out feature extraction, the first feature map and the second feature map corresponding to each first high-frequency feature map are obtained, the feature map splicing is carried out to the first feature map and the second feature map, the third feature map is obtained, and the K third feature maps are input to the second UNet network to obtain the blood vessel segmentation image corresponding to the K second fundus images. The high-frequency component of the fundus image is used to guide the cascade UNet grid to extract the blood vessel information gradually, the robustness and generalization ability of the blood vessel segmentation are enhanced, and the accuracy of the blood vessel segmentation is improved.

[0125] The first fundus image is extracted in high-frequency component, and a second high-frequency feature map is obtained. High-frequency loss calculation is performed according to the K first high-frequency feature maps and the second high-frequency feature map, and a high-frequency loss value is obtained. Segmentation loss calculation is performed according to the K blood vessel segmentation images and the blood vessel segmentation label, and a segmentation loss value is obtained. The high-frequency loss value and the segmentation loss value are added to obtain a target loss value. The network parameters of the first UNet network and the second UNet network are updated according to the target loss value, so as to train the first UNet network and the second UNet network.

[0126] The target fundus image is obtained, and the target fundus image is input to the trained first UNet network to reduce the style feature, and a target feature map is obtained. The target feature map is input to the trained second UNet network for image segmentation, and a target blood vessel segmentation image is obtained.

[0127] Please refer to Figure 7 The embodiment of the application also provides a medical image segmentation device, which can realize the medical image segmentation method described above. The device comprises:

[0128] The acquisition module 710 is configured to acquire a source domain medical image and a target domain medical image.

[0129] The style conversion module 720 is configured to convert the source domain medical image and the target domain medical image in style, and obtain a style transfer feature map of the target domain medical image.

[0130] The first feature extraction module 730 is configured to extract high-frequency features from the style transfer feature map, and obtain a first high-frequency feature map.

[0131] The second feature extraction module 740 is configured to extract features from the first high-frequency feature map by using a preset first UNet network, and obtain a first feature map and a second feature map.

[0132] The feature splicing module 750 is configured to splice the first feature map and the second feature map to obtain a third feature map.

[0133] The medical image segmentation module 760 is configured to perform image segmentation processing on the third feature map by using a preset second UNet network to obtain an image segmentation result of the target domain medical image.

[0134] The specific implementation of the medical image segmentation device is basically the same as the specific embodiments of the medical image segmentation method described above, and will not be repeated here.

[0135] The embodiments of the present application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the medical image segmentation method described above. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0136] Please refer to Figure 8 , Figure 8 The hardware structure of the electronic device of another embodiment is illustrated, which includes:

[0137] The processor 810 can be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0138] The memory 820 can be implemented in the form of a ROM (ReadOnly Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory). The memory 820 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 820 and are called and executed by the processor 810 to implement the medical image segmentation method of the embodiments of the present application.

[0139] The input / output interface 830 is used to realize information input and output.

[0140] The communication interface 840 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0141] A bus 850 transmits information between the various components (for example, the processor 810, the memory 820, the input / output interface 830, and the communication interface 840) of the device.

[0142] The processor 810, the memory 820, the input / output interface 830, and the communication interface 840 are communicatively connected to each other within the device through the bus 850.

[0143] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the medical image segmentation method.

[0144] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0145] The medical image segmentation method, the medical image segmentation device, the electronic device, and the computer readable storage medium provided by the embodiment of the present application can obtain a source domain medical image and a target domain medical image, perform style conversion on the source domain medical image and the target domain medical image, and obtain a style transfer feature map of the target domain medical image. Through image style conversion, the target domain medical image can retain the original image content while obtaining the image style of the source domain medical image, thereby reducing the feature difference between the source domain medical image and the target domain medical image. Further, high-frequency feature extraction is performed on the style transfer feature map to obtain a first high-frequency feature map. Through high-frequency feature extraction, noise in the medical image can be reduced, and the feature difference between the source domain medical image content and the target domain medical image content in the style transfer feature map can be further reduced, thereby reducing the unique style features of the data domain. Furthermore, a first UNet network is preset to perform feature extraction on the first high-frequency feature map to obtain a first feature map and a second feature map. In order to avoid information loss, the first feature map and the second feature map are spliced to obtain a third feature map. Finally, a second UNet network is preset to perform image segmentation processing on the third feature map to obtain an image segmentation result of the target domain medical image, thereby improving the accuracy of medical image segmentation.

[0146] The embodiments described in the specification are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0147] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0148] The device embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0149] Those skilled in the art can understand that all or some steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0150] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0151] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" or the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b and c can be single or multiple.

[0152] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0153] The units described above as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0154] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0155] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0156] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method of medical image segmentation, characterized by, The method comprises: obtaining source domain medical images and target domain medical images; wherein the source domain medical images have image segmentation labels, and the target domain medical images do not have image segmentation labels; performing style conversion on the source domain medical images and the target domain medical images to obtain a style transfer feature map of the target domain medical images; performing high-frequency feature extraction on the style transfer feature map to obtain a first high-frequency feature map; performing feature extraction on the first high-frequency feature map through a preset first UNet network to obtain a first feature map and a second feature map; performing feature map splicing on the first feature map and the second feature map to obtain a third feature map; performing image segmentation processing on the third feature map through a preset second UNet network to obtain an image segmentation result of the target domain medical images; The method comprises: performing image domain transformation on the source domain medical images to obtain a first image frequency domain map of the source domain medical images, and performing image domain transformation on the target domain medical images to obtain a second image frequency domain map of the target domain medical images; wherein the first image frequency domain map comprises a first amplitude map and a first phase map, and the second image frequency domain map comprises a second amplitude map; performing image fusion on the first amplitude map and the second amplitude map to obtain a target amplitude map; performing frequency domain map construction according to the target amplitude map and the first phase map to obtain a target frequency domain map; performing image domain restoration on the target frequency domain map to obtain the style transfer feature map; The method comprises: performing Gaussian filtering processing on the style transfer feature map to obtain a filtered feature map; performing image difference on the style transfer feature map and the filtered feature map to obtain the first high-frequency feature map.

2. The medical image segmentation method according to claim 1, characterized in that, The method comprises: performing Fourier transform on the source domain medical images to obtain the first image frequency domain map, and performing Fourier transform on the target domain medical images to obtain the second image frequency domain map.

3. The medical image segmentation method according to claim 1, characterized in that, The first UNet network comprises a first double convolution layer and a second double convolution layer, and the first feature map and the second feature map are obtained by performing feature extraction on the first high-frequency feature map through the preset first UNet network, which comprises: performing feature extraction on the first high-frequency feature map through the first double convolution layer to obtain the first feature map; performing feature extraction on the first feature map through the second double convolution layer to obtain the second feature map.

4. The medical image segmentation method according to claim 3, characterized in that, The first double convolution layer comprises a first convolution layer, a first batch normalization layer, a first activation layer, a second convolution layer, a second batch normalization layer, and a second activation layer, and the first feature map is obtained by performing feature extraction on the first high-frequency feature map through the first double convolution layer, which comprises: The first high-frequency feature map is subjected to convolution processing by the first convolution layer to obtain a first convolution feature map, the first convolution feature map is subjected to batch regularization processing by the first batch normalization layer to obtain a second convolution feature map, and the second convolution feature map is subjected to activation processing by the first activation layer to obtain an activation feature map; The activation feature map is subjected to convolution processing by the second convolution layer to obtain a third convolution feature map, the third convolution feature map is subjected to batch regularization processing by the second batch normalization layer to obtain a fourth convolution feature map, and the fourth convolution feature map is subjected to activation processing by the second activation layer to obtain the first feature map.

5. The medical image segmentation method according to any one of claims 1 to 4, characterized in that, After the third feature map is subjected to image segmentation processing by the preset second UNet network to obtain the image segmentation result of the target domain medical image, the medical image segmentation method further comprises: performing high-frequency feature extraction on the source domain medical image to obtain a second high-frequency feature map; performing first loss calculation on the second feature map and the second high-frequency feature map to obtain high-frequency loss data; performing second loss calculation on the image segmentation label of the source domain medical image and the image segmentation result to obtain segmentation loss data; obtaining target loss data according to the high-frequency loss data and the segmentation loss data; updating network parameters of the first UNet network and the second UNet network according to the target loss data.

6. A medical image segmentation apparatus characterized by comprising: The device comprises: an acquisition module configured to acquire a source domain medical image and a target domain medical image; wherein the source domain medical image has an image segmentation label, and the target domain medical image does not have an image segmentation label; a style conversion module configured to perform style conversion on the source domain medical image and the target domain medical image to obtain a style transfer feature map of the target domain medical image; a first feature extraction module configured to perform high-frequency feature extraction on the style transfer feature map to obtain a first high-frequency feature map; a second feature extraction module configured to perform feature extraction on the first high-frequency feature map by a preset first UNet network to obtain a first feature map and a second feature map; a feature splicing module configured to splice the first feature map and the second feature map to obtain a third feature map; a medical image segmentation module configured to perform image segmentation processing on the third feature map by a preset second UNet network to obtain an image segmentation result of the target domain medical image; the device is further configured to: perform image domain transformation on the source domain medical image to obtain a first image frequency domain map of the source domain medical image, and perform image domain transformation on the target domain medical image to obtain a second image frequency domain map of the target domain medical image; wherein the first image frequency domain map comprises a first amplitude map and a first phase map, and the second image frequency domain map comprises a second amplitude map; perform image fusion on the first amplitude map and the second amplitude map to obtain a target amplitude map; perform frequency domain map construction according to the target amplitude map and the first phase map to obtain a target frequency domain map; perform image domain restoration on the target frequency domain map to obtain the style transfer feature map. performing Gaussian filtering on the style transfer feature map to obtain a filtered feature map; performing image subtraction on the style transfer feature map and the filtered feature map to obtain the first high-frequency feature map.

7. An electronic device, characterized by The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the medical image segmentation method in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the medical image segmentation method in any one of claims 1 to 5.

Images