Fetal Brain Tissue Segmentation Method, Device, Medium and Terminal Based on Cycle-Consistent Network

Abstract

The present invention provides a fetal brain tissue segmentation method, device, medium and terminal based on a cycle consistency network. By acquiring target fetal brain image data, inputting the target fetal brain image into a pre-trained cycle consistency network, and outputting a segmentation result of the target fetal brain image corresponding to the gestational week; The present invention adopts a learning framework combining cycle consistency and adversarial learning, and realizes the accurate capture of fine-grained domain-invariant brain structures in thick clinical and thin reconstruction cases of fetal brain MR images.

Description

Technical Field

[0001] The present invention relates to the field of medical images, and particularly to a fetal brain tissue segmentation method, device, medium and terminal based on a cycle consistency network. Background Art

[0002] Magnetic resonance imaging (MRI) is a very important medical image imaging technology, which has the characteristics of low radiation, high sensitivity for soft tissue imaging, multi-directional imaging, and multiple presentation methods. In order to measure the brain growth pattern and morphological changes of neurodevelopmental disorders or separate the lesion sites, it is necessary to segment MRI images. Image segmentation, as a general digital image analysis technology, plays a crucial role in the medical field.

[0003] However, due to the lack of high-quality labels, automatically and accurately segmenting fetal brain tissue from clinical thick MR images is a challenging task. For example, there is no obvious visual boundary between the cortical plate and cerebrospinal fluid. This brings great uncertainty to different samples, and it is difficult for even experienced radiologists to delineate. In the prior art, deep neural networks in medical image segmentation methods mainly focus on the reconstruction of high-resolution MR images with high-quality annotations, which has an obvious domain gap with the MR images directly collected clinically. In addition, unsupervised domain adaptation (UDA) segmentation networks have great limitations in multi-label segmentation tasks. For categories with similar features, they cannot effectively distinguish them only using unsupervised methods, resulting in chaotic prediction results for similar tissues and it is difficult to achieve accurate segmentation results. Summary of the Invention

[0004] In view of the above-mentioned disadvantages of the prior art, the purpose of the present invention is to provide a fetal brain tissue segmentation method, device, medium and terminal based on a cycle consistency network, which is used to solve the problem of automatically and accurately extracting brain tissue from unlabeled clinical fetal brain images in the prior art.

[0005] To achieve the above object and other related objects, the present invention provides a fetal brain tissue segmentation method based on a cycle consistency network, which is characterized in that the method includes: obtaining target fetal brain image data; inputting the target fetal brain image into a pre-trained cycle consistency network, and outputting a segmentation result of the target fetal brain image corresponding to the gestational week.

[0006] In an embodiment of the present invention, the cycle consistency network includes:

[0007] A processing module for inputting the target fetal brain image and outputting a high-frequency image and a low-frequency image; a segmentation network connected to the processing module for inputting the high-frequency image and outputting a predicted target fetal brain segmentation result; a generation network connected to the segmentation network for inputting the predicted target fetal brain segmentation result and the low-frequency image and outputting the target fetal brain image segmentation result corresponding to the gestational week.

[0008] In an embodiment of the present invention, the cycle consistency network further includes: a discriminator and a segmenter in the segmentation network, and a generator in the generation network

[0009] In an embodiment of the present invention, the method for the processing module to process the target fetal brain image includes: based on frequency-domain invariant information, performing a transformation on the fetal brain image to extract a high-frequency image; based on frequency-domain specific information, performing a transformation on the fetal brain image to extract a low-frequency image.

[0010] In an embodiment of the present invention, the cycle consistency network is obtained by training with multiple brain image segmentation training samples; wherein, the image segmentation training samples include source domain image data samples composed of multiple high-resolution brain image samples and registered high-resolution brain image samples, and target domain image data samples composed of multiple fetal brain image samples and their corresponding manually segmented fetal brain image samples.

[0011] In an embodiment of the present invention, the acquisition method of the source domain data sample includes: acquiring a high-resolution brain image, and registering the high-resolution image based on the fetal brain image with the closest gestational week to obtain a registered high-resolution image.

[0012] To achieve the above object and other related objects, the present invention provides a fetal brain tissue segmentation device based on a cycle consistency network, characterized in that the device includes: an acquisition device for acquiring target fetal brain image data; a cycle consistency device for inputting the target fetal brain image into a pre-trained cycle consistency network and outputting the target fetal brain image segmentation result corresponding to the gestational week.

[0013] In an embodiment of the present invention, the cycle consistency network includes: a processing module for inputting the target fetal brain image and outputting a high-frequency image and a low-frequency image; a segmentation network connected to the processing module for inputting the high-frequency image and outputting a predicted target fetal brain segmentation result; a merging module connected to the processing module and the segmenter for inputting the predicted target fetal brain image segmentation result and the low-frequency image and outputting a merged predicted target fetal brain image segmentation result; a generation network connected to the segmentation network for inputting the predicted target fetal brain image segmentation result and outputting the target fetal brain image segmentation result corresponding to the gestational week.

[0014] To achieve the above object and other related objects, the present invention provides a computer-readable storage medium, characterized in that it stores computer instructions, and when the computer instructions are run, the above-mentioned method is executed.

[0015] To achieve the above object and other related objects, the present invention provides an electronic terminal, characterized in that the device includes: a memory and a processor; the memory is used to store computer instructions; the processor runs the computer instructions to implement the above-mentioned method.

[0016] As described above, the fetal brain tissue segmentation method, device, medium and terminal based on the cycle consistency network of the present invention have the following beneficial effects: Different from the existing deep learning segmentation algorithms and unsupervised domain adaptation segmentation networks, the fetal brain tissue segmentation method, device, medium and terminal based on the cycle consistency network provided by the present application designs a cycle consistency network in unsupervised domain adaptation. Compared with the existing technology, the present application constructs a new source data as the input of the segmentation network through registration, provides more structure-invariant information for the segmentation network, and reduces the differences in brain tissue development during pregnancy; at the same time, a new unsupervised domain adaptation framework based on frequency domain invariant (i.e., Fourier content coding (FCC)) and frequency domain specific (i.e., Fourier style coding (FSC)) information is designed, which facilitates the cycle consistency network to fully perceive the deviation, adjust the optimization strategy, and realize self-supervision of unannotated clinical thick MR images while making the segmentation results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It shows a schematic flowchart of a fetal brain tissue segmentation method based on a cycle consistency network in an embodiment of the present invention.

[0018] Figure 2 It shows a schematic diagram of the cycle consistency network structure in an embodiment of the present invention.

[0019] Figure 3 It shows a schematic diagram of the Fourier encoding result in an embodiment of the present invention.

[0020] Figure 4 It shows a schematic diagram of the detailed parameter settings of the segmenter, generator and discriminator in an embodiment of the present invention.

[0021] Figure 5 It shows a schematic diagram of the registration of source data in an embodiment of the present invention.

[0022] Figure 6 It shows a schematic diagram of the source domain and target domain data in an embodiment of the present invention.

[0023] Figure 7It shows a schematic structural diagram of the cycle consistency network in an embodiment of the present invention.

[0024] Figure 8 It shows a schematic diagram of the fetal brain image segmentation result in an embodiment of the present invention.

[0025] Figure 9 It shows a schematic diagram of the fetal brain image segmentation results at different gestational weeks in an embodiment of the present invention.

[0026] Figure 10 It shows a schematic diagram of the ablation experiment results in an embodiment of the present invention.

[0027] Figure 11 It shows a schematic structural diagram of the fetal brain tissue segmentation device based on the cycle consistency network in an embodiment of the present invention.

[0028] Figure 12 It shows a schematic diagram of the terminal for fetal brain tissue segmentation based on the cycle consistency network in an embodiment of the present invention. Detailed implementation manners

[0029] The following specific embodiments illustrate the implementation manners of the present invention. Those skilled in this technology can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

[0030] Please refer to Figures 1 to 12 . It should be noted that the structures, ratios, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in the specification for those skilled in this technology to understand and read, and are not used to limit the limiting conditions under which the present invention can be implemented. Therefore, they do not have technical essential significance. Any modification of the structure, change of the proportional relationship, or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope that can be covered by the technical content disclosed in the present invention. At the same time, the terms such as "upper", "lower", "left", "right", "middle", and "one" cited in this specification are only for the convenience of clear narration, and are not used to limit the scope under which the present invention can be implemented. The change or adjustment of their relative relationships, without substantial change in the technical content, should also be regarded as the scope under which the present invention can be implemented.

[0031] Throughout this specification, unless otherwise clearly specified and limited, the terms "install", "connect", "link", "fix", "hold", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.

[0032] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms, unless the context clearly dictates otherwise. Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and in the above drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that the data used in this way may be interchanged where appropriate so that the embodiments described herein can be implemented in an order different from that shown or described herein. It should be further understood that the terms "comprising", "including" indicate the presence of the stated features, operations, elements, components, items, types, and / or groups, but do not preclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, types, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B and C", with the exception of this definition only when the combinations of elements, functions or operations are mutually exclusive in some way.

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the technical solutions in the embodiments of the present invention are further described in detail through the following embodiments in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0034] As Figure 1 A schematic flowchart showing a fetal brain tissue segmentation method based on a cycle consistency network in an embodiment of the present invention is presented.

[0035] The method includes:

[0036] S11: Obtain target fetal brain image data;

[0037] Specifically, the target fetal brain image data is magnetic resonance imaging (MRI). Preferably, preprocessing operations such as normalization, image enhancement, and denoising are performed on the target fetal brain image to remove interference factors. In the above embodiment of the present invention, the target fetal brain image data can be stored in a set device, and the set device can be network-connected to a fetal brain tissue segmentation device based on a cycle consistency network to obtain the target fetal brain image data. The above network connection can be a wireless or wired network connection. If the set device is communicatively connected to the fetal brain tissue segmentation device based on the cycle consistency network, the network mode of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, etc.

[0038] S12: Input the target fetal brain image into a pre-trained cycle consistency network and output a segmentation result corresponding to the gestational week.

[0039] Specifically, the gestational weeks of the target fetal images input each time are not necessarily the same. The fetal brain develops rapidly at different gestational weeks, and there are obvious differences in tissue structure. This developmental difference at different gestational weeks is an important factor affecting domain shift, and domain shift will affect the accuracy of the segmentation result of the final target fetal brain image. The pre-trained cycle consistency network can reduce the differences between brain tissues at different gestational weeks and can output a more accurate segmentation result corresponding to the gestational week of the input target fetal brain image. For example: input a fetal brain image of 20 weeks, and output a segmentation result of the 20-week fetal brain image that is closer to the manual segmentation result of the 20-week fetal brain image.

[0040] In one embodiment, the cycle consistency network includes:

[0041] As Figure 2 shown, a processing module 21 for inputting the target fetal brain image and outputting a high-frequency image and a low-frequency image; specifically, the low-frequency component and the high-frequency component in the target fetal brain image are extracted through Fourier encoding to form a corresponding low-frequency image and high-frequency image respectively.

[0042] In one embodiment, the processing method of the processing module for the target fetal brain image includes:

[0043] Based on the frequency-domain invariant information, transform and extract the fetal brain image to obtain a high-frequency image; based on the frequency-domain specific information, transform and extract the fetal brain image to obtain a low-frequency image. Specifically, according to Fourier encoding, decompose the target fetal brain image into the low-frequency components corresponding to the Fourier style coding (FSC) based on the frequency-domain specific (i.e., Fourier style coding (FSC)) information to obtain a low-frequency image; decompose the target fetal brain image into the high-frequency components corresponding to the Fourier style coding (FSC) based on the frequency-domain invariant (i.e., Fourier content coding (FCC)) information to obtain a high-frequency image. The low-frequency image retains a large amount of style information, while the high-frequency image, on the contrary, removes a large amount of style information and retains the structure with domain invariance. Among them, the function used in the process of decomposing the fetal brain image into the low-frequency components corresponding to the Fourier style coding (FSC) and the high-frequency components corresponding to the Fourier content coding (FCC) according to Fourier encoding is:

[0044] M α =(h, w, s)∈[-αH:αH, -αW:αW, -αS:αS]

[0045] FCC = F -1 ((1 - M α ) * F(x))

[0046] FSC = F -1 (M α * F(x))

[0047] Among them, given a fetal brain x of size H*W*S, we define a binary mask M α , which has all values equal to 0 except for the center position which is 1, and use the FFT algorithm to extract its frequency map. The extraction of FCC and FSC is shown in the above formula. Figure 3 shows the influence of α on the results in Fourier encoding, as well as the appearances of FSC and FCC for different values.

[0048] In an embodiment, the cycle consistency network includes:

[0049] A segmentation network 22, connected to the processing module 21, for the input high-frequency image, outputs a predicted target fetal brain segmentation result. Specifically, the high-frequency image highlights the contours, noises, and details of the fetal brain image. The predicted target fetal brain segmentation result obtained by inputting the high-frequency image that retains the domain-invariant structure into the segmentation network can retain more key information of the target fetal image than the predicted target fetal brain segmentation result obtained by directly inputting the fetal brain image into the segmentation network, reducing the loss of key information of the target fetal image caused during the operation and transformation process.

[0050] The generation network 23 is connected to the segmentation network 22 and is used to input the predicted segmentation result of the target fetal brain image and the low-frequency image, and output the segmentation result of the target fetal brain image corresponding to the gestational week; specifically, the generation network provides cyclic consistency constraints.

[0051] In one embodiment, the cyclic consistency network further includes:

[0052] The discriminator 202 and the segmenter 212 in the segmentation network, and the generator 203 in the generation network; specifically, the discriminator 202 is connected to the segmenter 212, and the segmenter 212 is connected to the generator 203; the high-frequency image is input into the discriminator mainly to achieve feature space alignment, and the high-frequency image after passing through the discriminator is input into the segmenter to obtain the predicted high-frequency image segmentation result.

[0053] In one embodiment, the training method of the cyclic consistency network includes:

[0054] The cyclic consistency network is obtained by training with multiple brain image segmentation training samples; among them, the image segmentation training samples include source domain image data samples composed of multiple high-resolution brain image samples and registered high-resolution brain image samples, and target domain image data samples composed of multiple fetal brain image samples and their corresponding manually segmented fetal brain image samples;

[0055] Specifically, the segmentation network and the generation network are obtained by training with brain image segmentation training samples. There is an obvious visual boundary between the cortical plate and cerebrospinal fluid in the high-resolution brain image, and its segmentation result is already clear; the source domain image and the target domain image are obtained and input into the processing module to obtain the corresponding high-frequency source domain image, low-frequency source domain image, high-frequency target domain image and low-frequency target domain image. Each time the source domain image obtained may be a high-resolution image or a registered high-resolution image, and the target domain image input each time is a fetal brain image; the high-frequency source domain image and the high-frequency target domain image are input into the discriminator to obtain the corresponding class target domain image and class source domain image respectively, and the loss of the discriminator is calculated based on the class target domain image, class source domain image and fetal brain image This step realizes feature space alignment; the class target domain image and the class source domain image are input into the segmenter to obtain the corresponding predicted source domain image segmentation result and predicted target domain image segmentation result respectively, and the loss L of the discriminator is calculated based on the predicted source domain image segmentation result, source domain image, predicted target domain image segmentation result and fetal brain image segmentation result advIn this process, due to the use of high-frequency images with domain-invariant structures, the contours, noise and details of the image are highlighted, thereby enhancing the computational effect of the segmentation network; the predicted source domain image segmentation results and the predicted target domain image segmentation results are combined with the low-frequency source domain image and the low-frequency target domain image to obtain the synthetically predicted source domain image segmentation results and the synthetically predicted target domain image segmentation results; the synthetically predicted source domain image segmentation results and the synthetically predicted target domain image segmentation results are input into the generator to obtain the source domain image segmentation results and the target domain image segmentation results corresponding thereto respectively. The use of low-frequency source domain images and low-frequency target domain images is mainly to provide style coding for the generator so that the generator can generate data in two different domains. The low-frequency images generated based on FSC are mainly for the generator. Provide style coding, and FSC can achieve the role of style coding in a simple and effective way, wherein the generator provides cycle consistency constraints, and the data from the two domains will obtain corresponding segmentation results. It is hoped that the generation network can generate data in two different domains, so style coding must be provided for the generation network. The low-frequency image generated based on FSC provides style coding for the generation network, which retains a large amount of style information, and FCC is particularly sensitive to the transformation of image pixel values, which just makes up for the disadvantage of poor contrast between low-resolution image tissues. Therefore, the method of this application can achieve good results in the segmentation of single-modal domain migration tasks. The loss of the generator is calculated based on the corresponding source domain image segmentation results, target domain image segmentation results, source domain image and fetal brain tissue manual segmentation results. like Figure 4 The parameters of the segmenter, generator, and discriminator are adjusted during the training process shown, and the loss function of the entire cycle-consistent network is:

[0056]

[0057] Among them, γ and β are introduced as hyperparameters, preferably, γ = 3.0 and β = 0.1.

[0058] In one embodiment, the method of obtaining the source domain image data sample includes:

[0059] like Figure 5 As shown in FIG, a high-resolution image is obtained, and the high-resolution image is registered based on the fetal brain image closest to the gestational age to obtain a registered high-resolution image; specifically, the fetal brain of different gestational ages develops rapidly, and the tissue structure has obvious differences. We regard this developmental difference at different gestational ages as part of the domain shift, so we use the registration to reduce this difference. This solves the problem that traditional domain adaptation methods have little improvement in fetal brain tissue segmentation, such as Figure 6As shown, register the high-resolution image in the source domain data sample to the target domain data sample, select the target domain image data sample closest to the desired gestational week (e.g., 25 weeks) in the target domain data sample, and obtain the registered high-resolution image corresponding to the gestational week. And so on, the registered high-resolution image can also be registered to the target domain data sample, select the target domain image data sample closest to the desired gestational week (e.g., 20 weeks) in the target domain data sample, and obtain the registered high-resolution image corresponding to the gestational week here. The size and shape of the obtained registered high-resolution image have changed, but the boundary between the cortical plate and cerebrospinal fluid in the image still has an obvious visual boundary. A new source domain image data is constructed as the input of the segmentation network. Through this operation step, more structure-invariant information is provided for the segmentation network; in addition, the number of the source domain data sample and the target domain image data sample may not be the same, and the input of the network does not require paired source domain image data and target domain image data, which enables the network to flexibly handle various new data.

[0060] To better illustrate the above image synthesis and segmentation method, the present invention provides the following specific embodiments.

[0061] Embodiment 1: A fetal brain tissue segmentation method based on a cycle consistency network. Figure 7 Shown is a structural schematic diagram of a fetal brain tissue segmentation method based on a cycle consistency network.

[0062] Obtain target fetal brain image data;

[0063] Input the target fetal brain image into a pre-trained cycle consistency network, and output the segmentation result of the target fetal brain image corresponding to the gestational week;

[0064] Among them, the overall framework for implementing the fetal brain tissue segmentation method includes a segmentation network (S) and a generator network (G); in the overall network training, a small group of annotated reconstructed high-resolution MR images (source data) are used to guide the brain tissue segmentation on unannotated clinical thick MR images (target data). 1) Register the source data images to the target data images of the closest gestational age. The registered source data images and the original source data images together form a new source data image dataset. Here, the new source data images and the target data images do not have to be input into the network in pairs. All the images will ultimately be input into the network to learn as much information as possible, achieving the construction of a new source data based on registration as the input of the segmentation network, providing more structure-invariant information for the segmentation network. The differences between gestational ages are considered part of the domain gap and can be alleviated by registering to the corresponding gestational age data. 2) The process of decomposing the input source domain image data and the target domain image data into the low-frequency components corresponding to FSC and the high-frequency components corresponding to FCC. A novel unsupervised domain adaptation framework based on frequency-domain invariant (i.e., Fourier content coding (FCC)) and frequency-domain specific (i.e., Fourier style coding (FSC)) information plays a crucial role in solving the single-modal domain transfer problem. 3) The segmentation network and the generation network in the framework form a cycle. The generation network provides cycle consistency constraints. Different from Cycle-GAN, our framework combines cycle consistency with adversarial learning to perform conversions between MR images and their segmentation maps, capable of capturing fine-grained domain-invariant brain structures in thick clinical and thin reconstruction cases of fetal brain MR images.

[0065] We collected multiple brain image segmentation samples in clinical practice to verify the proposed cycle consistency network. All experiments were implemented using PyTorch on an RTX3060 GPU. For data preprocessing, we first rotated the original MR images at any angle from 0° to 360°, and then resized them to scans with a block size of 128×192 as the input of the segmentation network. All images were preprocessed by bias field correction and normalized. During the training process, the learning rate was 0.00001, and the Adam optimizer was used to optimize all parameters. Figure 8 Shows the quantitative results and visual comparisons of the method of this application and the currently commonly used UDA algorithms. The method of this application is the best in all cases. In addition, Figure 9 Also shows the performance of our method and other methods in comparison on different gestational age data. Our method is also significantly better. Finally, the results of the ablation experiment part of the method are as Figure 10 shown, which can clearly demonstrate the improvement of each key module on the network performance.

[0066] As Figure 11As shown, it presents a schematic structural diagram of a fetal brain tissue segmentation device based on a cycle consistency network in an embodiment of the present invention. In this embodiment, as Figure 11 As shown, it presents a schematic structural diagram of a fetal brain tissue segmentation device based on a cycle consistency network in an embodiment of the present invention. In this embodiment, the fetal brain tissue segmentation device based on a cycle consistency network includes:

[0067] An acquisition device 111, configured to acquire target fetal brain image data;

[0068] A cycle consistency device 112, configured to input the target fetal brain image into a pre-trained cycle consistency network and output a segmentation result of the target fetal brain image corresponding to the gestational week.

[0069] In one embodiment, the cycle consistency network includes:

[0070] A processing module, configured to input the target fetal brain image and output a high-frequency image and a low-frequency image; a segmentation network, connected to the processing module, configured to input the high-frequency image and output a predicted target fetal brain segmentation result; a generation network, connected to the segmentation network, configured to input the predicted target fetal brain segmentation result and the low-frequency image and output the segmentation result of the target fetal brain image corresponding to the gestational week.

[0071] The fetal brain tissue segmentation method based on a cycle consistency network provided by an embodiment of the present invention can be implemented on the terminal side or the server side. For the hardware structure of the fetal brain tissue segmentation terminal based on a cycle consistency network, please refer to [[ID=1 / 18]] Figure 11 As shown in, it is an optional hardware structure schematic diagram of a fetal brain tissue segmentation terminal 1100 based on a cycle consistency network provided by an embodiment of the present invention. The terminal 1100 can be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The fetal brain tissue segmentation terminal 1100 based on a cycle consistency network includes: at least one processor 1101, a memory 1102, at least one network interface 1104, and a user interface 1106. Each component in the device is coupled together through a bus system 11011. It can be understood that the bus system 11011 is used to realize the connection and communication between these components. The bus system 11011 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 11 all kinds of buses are labeled as the bus system.

[0072] Among them, the user interface 1106 may include a display, a keyboard, a mouse, a trackball, a click gun, a button, a button, a touchpad, or a touch screen, etc.

[0073] It can be understood that the memory 1102 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory). The memory described in the embodiments of the present invention is intended to include but not limited to these and any other suitable categories of memories.

[0074] The memory 1102 in the embodiments of the present invention is used to store various categories of data to support the operation of the fetal brain tissue segmentation terminal 1100 based on the cycle consistency network. Examples of these data include: any executable programs for operating on the fetal brain tissue segmentation terminal 1100 based on the cycle consistency network, such as the operating system 11021 and application programs 11022; the operating system 11021 contains various system programs, such as the framework layer, the core library layer, the driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 11022 can contain various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The fetal brain tissue segmentation method provided by the embodiments of the present invention can be included in the application program 11022.

[0075] The method disclosed in the embodiments of the present invention above can be applied to the processor 1101 or implemented by the processor 1101. The processor 1101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 1101 or instructions in the form of software. The above-mentioned processor 1101 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1101 can implement or execute each method, step, and logic block diagram disclosed in the embodiments of the present invention. The general-purpose processor 1101 may be a microprocessor or any conventional processor, etc. Combining the steps of the accessory optimization method provided in the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, and this storage medium is located in the memory. The processor reads the information in the memory and combines its hardware to complete the steps of the foregoing method.

[0076] In an exemplary embodiment, the fetal brain tissue segmentation terminal 800 based on the cycle consistency network can be one or more application-specific integrated circuits (ASICs, Application Specific Integrated Circuits), DSPs, programmable logic devices (PLDs, Programmable Logic Devices), complex programmable logic devices (CPLDs, Complex Programmable Logic Devices) for executing the foregoing method.

[0077] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to a computer program. The foregoing computer program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0078] In summary, a fetal brain tissue segmentation method, device, medium, and terminal based on a cycle consistency network according to the present invention obtain target fetal brain image data, input the target fetal brain image into a pre-trained cycle consistency network, and output a segmentation result of the target fetal brain image corresponding to the gestational week; the present invention adopts a learning framework that combines cycle consistency and adversarial learning, and realizes the precise capture of fine-grained domain-invariant brain structures in thick clinical and thin reconstruction cases of fetal brain MR images. Therefore, the present invention effectively overcomes various disadvantages in the prior art and has high industrial utilization value.

[0079] The above embodiments are only illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A fetal brain tissue segmentation method based on a cycle consistency network, characterized in that: The method comprises: Acquiring target fetal brain image data; Inputting the target fetal brain image into a pre-trained cycle consistency network, and outputting a target fetal brain image segmentation result corresponding to the gestational age; Wherein, the cycle consistency network includes: a processing module, configured to input the target fetal brain image and output a high-frequency image and a low-frequency image; A segmentation network, connected to the processing module, for inputting the high-frequency image and outputting the predicted target fetal brain image segmentation result; the discriminator and segmenter in the segmentation network, and the generator in the generation network; generating a network connected to the segmentation network, configured to input the predicted target fetal brain segmentation result and the low-frequency image, and output the target fetal brain image segmentation result corresponding to the gestational age; The processing module processes the target fetal brain image in a method comprising: transforming and extracting the target fetal brain image data based on frequency domain invariant information to obtain a high-frequency image; transforming and extracting the target fetal brain image data based on frequency domain specific information to obtain a low-frequency image; The loss function of the cycle consistency network is: represents the loss function of the discriminator based on the target domain image, the source domain image and the fetal brain image; L adv represents the loss function of the discriminator calculated based on the predicted source domain image segmentation result, the source domain image, the predicted target domain image segmentation result, and the fetal brain image segmentation result; represents the loss function of the generator calculated based on the corresponding source domain image segmentation results, target domain image segmentation results, source domain image and fetal brain tissue manual segmentation results; γ and β represent hyperparameters.

2. The fetal brain tissue segmentation method based on cycle consistency network according to claim 1, characterized in that: The cycle consistency network is obtained by training multiple brain image segmentation training samples; The image segmentation training samples include source domain image data samples consisting of multiple high-resolution brain image samples and registered high-resolution brain image samples, and target domain image data samples consisting of multiple fetal brain image samples and their corresponding fetal brain image manually segmented samples.

3. The fetal brain tissue segmentation method based on cycle consistency network according to claim 2, characterized in that: The method of obtaining the source domain image data sample includes: A high-resolution brain image is acquired, and the high-resolution brain image is registered based on a fetal brain image closest to the gestational age to acquire a registered high-resolution image.

4. A fetal brain tissue segmentation device based on a cycle consistency network, characterized in that: The device comprises: an acquisition device for acquiring target fetal brain image data; a cycle consistency device, configured to input the target fetal brain image into a pre-trained cycle consistency network and output a target fetal brain image segmentation result corresponding to the gestational age; Wherein, the cycle consistency network includes: a processing module, configured to input the target fetal brain image and output a high-frequency image and a low-frequency image; A segmentation network, connected to the processing module, for inputting the high-frequency image and outputting the predicted target fetal brain image segmentation result; the discriminator and segmenter in the segmentation network, and the generator in the generation network; generating a network connected to the segmentation network, configured to input the predicted target fetal brain segmentation result and the low-frequency image, and output the target fetal brain image segmentation result corresponding to the gestational age; The processing module processes the target fetal brain image in a method comprising: transforming and extracting the target fetal brain image data based on frequency domain invariant information to obtain a high-frequency image; transforming and extracting the target fetal brain image data based on frequency domain specific information to obtain a low-frequency image; The loss function of the cycle consistency network is: represents the loss function of the discriminator based on the target domain image, the source domain image and the fetal brain image; L adv represents the loss function of the discriminator calculated based on the predicted source domain image segmentation result, the source domain image, the predicted target domain image segmentation result, and the fetal brain image segmentation result; represents the loss function of the generator calculated based on the corresponding source domain image segmentation results, target domain image segmentation results, source domain image and fetal brain tissue manual segmentation results; γ and β represent hyperparameters.

5. The fetal brain tissue segmentation device based on cycle consistency network according to claim 4, characterized in that: The cycle consistency network includes: a processing module, configured to input the target fetal brain image and output a high-frequency image and a low-frequency image; A segmentation network, connected to the processing module, configured to input the high-frequency image and output a predicted target fetal brain image segmentation result; A network is generated, connected to the segmentation network, and is configured to input the predicted target fetal brain segmentation result and the low-frequency image, and output the target fetal brain image segmentation result corresponding to the gestational age.

6. A computer-readable storage medium, characterized in that Computer instructions are stored, and when the computer instructions are executed, the method according to any one of claims 1 to 3 is performed.

7. An electronic terminal, characterized in that: The electronic terminal includes: a memory and a processor; the memory is used to store computer instructions; the processor executes the computer instructions to implement the method according to any one of claims 1 to 3.

Images