Image segmentation method, equipment and device for nuclear magnetic resonance image

Abstract

The invention relates to the field of image processing, in particular to an image segmentation method, equipment and device for a nuclear magnetic resonance image, and aims to provide a scheme for conveniently processing nuclear magnetic resonance images with different resolutions while ensuring a segmentation result of the nuclear magnetic resonance image of a brain structure. According to the embodiment of the invention, the method comprises the steps: obtaining a nuclear magnetic resonance image of a human head, and inputting the nuclear magnetic resonance image into a trained segmentation model; based on the trained segmentation model, performing feature extraction on the nuclear magnetic resonance image, and determining a feature map including image features of the nuclear magnetic resonance image; according to a resolution conversion parameter determined in the training process of the trained segmentation model, carrying out dimension conversion processing on the feature map; performing up-sampling processing on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; and presenting the target nuclear magnetic resonance image in a display interface, and identifying different brain partitions in the target nuclear magnetic resonance image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image segmentation method, equipment and device for magnetic resonance imaging. Background Art

[0002] Artificial intelligence has developed rapidly, and has made breakthrough achievements in the field of image processing. Deep learning methods have also made some progress in brain segmentation. However, deep learning network methods generally only support a fixed resolution (usually 1.0mm) and have limited processing capabilities for images of different resolutions.

[0003] Therefore, when using deep learning network methods to process MRI images containing brain structures, only MRI images with fixed resolution can be processed, and the existing MRI image processing methods are very limited. Summary of the invention

[0004] The purpose of the present application is to provide a method, device and apparatus for image segmentation of a nuclear magnetic resonance image, so as to provide a solution for conveniently processing nuclear magnetic resonance images of different resolutions while ensuring the segmentation results of the nuclear magnetic resonance images of brain structures.

[0005] In a first aspect, the present application provides a method for image segmentation of a nuclear magnetic resonance image, the method comprising:

[0006] Acquire a nuclear magnetic resonance image of a human head, and input the nuclear magnetic resonance image into a trained segmentation model;

[0007] Based on the trained segmentation model, feature extraction is performed on the nuclear magnetic resonance image to determine a feature map including image features of the nuclear magnetic resonance image; according to the resolution conversion parameter determined in the training process of the trained segmentation model, dimension conversion processing is performed on the feature map; up-sampling processing is performed on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions;

[0008] The target MRI image is presented in a display interface, and different brain regions in the target MRI image are identified.

[0009] In a second aspect, the present application provides an electronic device, the device comprising at least one processor and at least one memory;

[0010] The memory stores program codes, and when the program codes are executed by the processor, the processor performs the following process:

[0011] Acquire a nuclear magnetic resonance image of a human head, and input the nuclear magnetic resonance image into a trained segmentation model;

[0012] Based on the trained segmentation model, feature extraction is performed on the nuclear magnetic resonance image to determine a feature map including image features of the nuclear magnetic resonance image; according to the resolution conversion parameter determined in the training process of the trained segmentation model, dimension conversion processing is performed on the feature map; up-sampling processing is performed on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions;

[0013] The target MRI image is presented in a display interface, and different brain regions in the target MRI image are identified.

[0014] In a third aspect, the present application provides an image segmentation device for a nuclear magnetic resonance image, comprising:

[0015] An acquisition module, used for acquiring a nuclear magnetic resonance image of a human head, and inputting the nuclear magnetic resonance image into a trained segmentation model;

[0016] a segmentation module, which extracts features from the nuclear magnetic resonance image based on the trained segmentation model to determine a feature map including image features of the nuclear magnetic resonance image; performs dimension conversion processing on the feature map according to a resolution conversion parameter determined in the training process of the trained segmentation model; performs upsampling processing on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions;

[0017] The presentation module is used to present the target MRI image in a display interface and identify different brain regions in the target MRI image.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to perform the image segmentation method of magnetic resonance images as described in the first aspect above.

[0019] In a fifth aspect, the present application provides a computer program product, including a computer program:

[0020] When the computer program is executed by a processor, the image segmentation method of the nuclear magnetic resonance image as described in the first aspect above is implemented.

[0021] After the embodiment of the present application obtains the nuclear magnetic resonance image of the human head, the nuclear magnetic resonance image is input into the trained segmentation model, and then based on the trained segmentation model, the nuclear magnetic resonance image is feature extracted to determine a feature map including the image features of the nuclear magnetic resonance image; according to the resolution conversion parameters determined in the training process of the trained segmentation model, the feature map is dimensionally converted. The embodiment of the present application performs dimension conversion processing on the feature map of the nuclear magnetic resonance image through the resolution conversion parameters, so that the trained segmentation model has the ability to process nuclear magnetic resonance images of different resolutions, can more conveniently process nuclear magnetic resonance images of different resolutions, improves the efficiency of processing nuclear magnetic resonance images, and can better preserve the image features in the nuclear magnetic resonance image. In addition, after the feature map is dimensionally converted based on the resolution conversion parameters, the feature map after the dimension conversion processing is upsampled to obtain the target nuclear magnetic resonance image, thereby ensuring that the target nuclear magnetic resonance image has more image features and that the target nuclear magnetic resonance image has more structural details. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 This is an optional application scenario of the embodiment of the present application;

[0024] Figure 2 This is another optional application scenario of the embodiment of the present application;

[0025] Figure 3 This is a flow chart of an image segmentation method for a nuclear magnetic resonance image according to an embodiment of the present application;

[0026] Figure 4 This is a schematic diagram of a nuclear magnetic resonance image corresponding to multiple planes according to an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a segmentation model according to an embodiment of the present application;

[0028] Figure 6 This is a schematic diagram of dimensionality reduction according to an embodiment of the present application;

[0029] Figure 7 This is a schematic diagram of a dimension upgrade according to an embodiment of the present application;

[0030] Figure 8 A target magnetic resonance image corresponding to multiple planes according to an embodiment of the present application;

[0031] Fig. 9 A logical schematic diagram of determining a three-dimensional nuclear magnetic resonance image according to an embodiment of the present application;

[0032] Fig.10 Another logical schematic diagram of determining a three-dimensional nuclear magnetic resonance image according to an embodiment of the present application;

[0033] Fig.11 This is a schematic diagram of a segmentation effect of an embodiment of the present application;

[0034] Fig.12 This is a flow chart of a training segmentation model according to an embodiment of the present application;

[0035] Fig.13 A schematic diagram of sample input during an initial training process in an embodiment of the present application;

[0036] Fig.14 This is a flow chart of determining a target sample according to an embodiment of the present application;

[0037] Fig.15 A schematic diagram of sample input in each training process other than the initial training process in an embodiment of the present application;

[0038] Fig.16 This is a schematic diagram of the overall process of a segmentation model training process according to an embodiment of the present application;

[0039] Fig.17 This is a schematic diagram of the structure of an image segmentation device for a nuclear magnetic resonance image according to an embodiment of the present application;

[0040] Fig.18 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0041] Fig.19 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0043] Furthermore, in the description of the embodiments of the present application, unless otherwise specified, “and” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “plurality” means two or more than two.

[0044] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0045] In order to facilitate understanding of the image segmentation method, device and apparatus of a nuclear magnetic resonance image provided in the embodiments of the present application, some terms in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0046] The softmax function, also known as the normalized exponential function, is used to present the results of multi-classification in the form of probabilities. It is widely used in multi-classification scenarios. It maps some inputs to real numbers between 0 and 1, and normalizes them to ensure that the sum is 1, so the sum of the probabilities of multi-classification is also exactly 1.

[0047] In order to make the purpose, technical solution and beneficial effects of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The image segmentation method of a nuclear magnetic resonance image in the embodiment of the present application can be applied to an electronic device, wherein the electronic device is a nuclear magnetic resonance apparatus 10 as an example for description. Figure 1 As shown, in an optional application scenario of an embodiment of the present application, the nuclear magnetic resonance apparatus 10 includes a display screen 101.

[0049] Based on the user's acquisition operation, the MRI apparatus 10 acquires a MRI image of a human head and inputs the MRI image into a trained segmentation model; based on the trained segmentation model, features are extracted from the MRI image to determine a feature map including image features of the MRI image; according to a resolution conversion parameter determined during the training of the trained segmentation model, a dimension conversion process is performed on the feature map; the feature map after the dimension conversion process is upsampled to obtain a target MRI image; the target MRI image is presented on the display screen 101, and different brain regions in the target MRI image are identified.

[0050] In addition, another optional application scenario of the embodiment of the present application is as follows: Figure 2 As shown, it includes a terminal device 20 and a server 21;

[0051] Among them, an image segmentation method of a nuclear magnetic resonance image in an embodiment of the present application can be applied to a terminal device 20 and a server 21, and wired transmission and / or wireless transmission can be performed between the terminal device 20 and the server 21.

[0052] The terminal device 20 is used to obtain a MRI image of a human head and send the MRI image to a server 21; receive a target MRI image sent by the server 21, present the target MRI image in a display interface, and identify different brain regions in the target MRI image.

[0053] The server 21 is used to extract features from the MRI image based on the trained segmentation model, and determine a feature map including image features of the MRI image; perform dimension conversion processing on the feature map according to a resolution conversion parameter determined in the training process of the trained segmentation model; perform upsampling processing on the feature map after the dimension conversion processing to obtain a target MRI image; the target MRI image includes a brain structure segmented from the MRI image, and the brain structure includes a plurality of brain regions; and send the target MRI image to the terminal device 20.

[0054] In the embodiment of the present application, the terminal device 20 is an electronic device used by the user, which may be a personal computer, a mobile phone, a tablet computer, a notebook, an intelligent voice interaction device, a medical imaging device, or other computer device with certain computing capabilities.

[0055] In the embodiment of the present application, the server 21 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, cloud functions, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0056] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 or Figure 2 The application scenario shown can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0057] like Figure 3 As shown, the present application embodiment provides an image segmentation method for a nuclear magnetic resonance image, and the specific steps are as follows:

[0058] Step S301, obtaining a nuclear magnetic resonance image of a human head, and inputting the nuclear magnetic resonance image into a trained segmentation model;

[0059] It should be noted that when the MRI machine collects MRI images of the patient's head, it will collect MRI images of the patient's head in three directions to obtain MRI images corresponding to three planes. Therefore, the multiple MRI images containing brain structures obtained in the embodiment of the present application will correspond to three planes respectively.

[0060] For example, the multiple obtained MRI images containing brain structures can correspond to MRI images of the axial plane, coronal plane, and sagittal plane of the patient's brain, respectively.

[0061] For example, the axial, coronal and sagittal MRI images can be obtained as follows: Figure 4 shown.

[0062] It should be noted that the magnetic resonance images input in the embodiment of the present application include magnetic resonance images of a patient corresponding to three planes respectively.

[0063] It should be noted that the embodiments of the present application can obtain nuclear magnetic resonance images from local storage, and can also obtain nuclear magnetic resonance images from other devices.

[0064] In an embodiment of the present application, after acquiring magnetic resonance images containing brain structures in different planes, the magnetic resonance images are input into a trained segmentation model corresponding to the planes corresponding to the multiple magnetic resonance images.

[0065] Optionally, the trained segmentation model includes three trained segmentation models corresponding to different planes.

[0066] Exemplarily, after acquiring multiple MRI images, if MRI image 1 corresponds to the axial plane, MRI image 2 corresponds to the coronal plane, and MRI image 3 corresponds to the sagittal plane, MRI image 1 is input into the trained segmentation model corresponding to the axial plane, MRI image 2 is input into the trained segmentation model corresponding to the coronal plane, and MRI image 3 is input into the trained segmentation model corresponding to the sagittal plane.

[0067] For any plane of the MRI image, perform the following operations based on the trained segmentation model corresponding to the plane:

[0068] Step S302: extracting features from the nuclear magnetic resonance image based on the trained segmentation model, and determining a feature map including image features of the nuclear magnetic resonance image;

[0069] Exemplarily, taking the trained segmentation model corresponding to the axial plane as an example, the encoder in the trained segmentation model corresponding to the axial plane is used to extract features of the magnetic resonance image of the axial plane to determine a feature map corresponding to the magnetic resonance image of the axial plane.

[0070] Step S303: performing dimension conversion processing on the feature map according to the resolution conversion parameters determined in the training process of the trained segmentation model;

[0071] Optionally, the resolution conversion parameters in the embodiment of the present application include resolution conversion parameters corresponding to the first resolution normalization module in the trained segmentation model, and resolution conversion parameters corresponding to the second resolution normalization module in the trained segmentation model.

[0072] The embodiment of the present application performs dimensionality reduction processing on the feature map according to the resolution conversion parameters corresponding to the first resolution normalization module.

[0073] The embodiment of the present application performs dimensionality increase processing on the feature map after dimensionality reduction processing according to the resolution conversion parameter corresponding to the second resolution normalization module.

[0074] Optionally, the trained segmentation model in the embodiment of the present application includes a resolution normalization module, wherein the resolution normalization module includes a first resolution normalization module and a second resolution normalization module.

[0075] The input of the first resolution normalization module is connected to the encoder, the output of the second resolution normalization module is connected to the decoder, and at least one encoder and / or at least one decoder are included between the first resolution normalization module and the second resolution normalization module;

[0076] Optionally, in an embodiment of the present application, according to the dimension of the feature map output by the first resolution normalization module, the output channel of the encoder connected to the input of the first resolution normalization module is adjusted, and the number of output channels of the encoder is adjusted to the number of channels corresponding to the dimension of the feature map output by the first resolution normalization module. Similarly, according to the dimension of the feature map input by the second resolution normalization module, the input channel of the decoder connected to the output of the second resolution normalization module is adjusted, and the number of input channels of the decoder is adjusted to the number of channels corresponding to the dimension of the feature map input by the second resolution normalization module.

[0077] For example, Figure 5 As shown in the figure, for the sake of convenience, the order of the encoder and decoder is from left to right. Figure 5 As shown, the input of the first resolution normalization module can be connected to the first encoder, the output of the second resolution normalization module can be connected to the last decoder, and the encoder and decoder are included between the first resolution normalization module and the second resolution normalization module.

[0078] It should be noted that the embodiment of the present application does not limit the number of encoders to which the first resolution normalization module is connected, and does not limit the number of decoders to which the second resolution normalization module is connected.

[0079] Optionally, in an embodiment of the present application, a first resolution normalization module is used to perform dimensionality reduction processing on a feature map output by the encoder based on a resolution conversion parameter corresponding to the first resolution normalization module;

[0080] In the first resolution normalization module of the embodiment of the present application, a corresponding scaling factor is determined according to a resolution conversion parameter corresponding to the first resolution normalization module, and based on the dimension of the feature map input into the first resolution normalization module and the scaling factor corresponding to the first resolution normalization module, a dimensionality reduction process is performed on the feature map input into the first resolution normalization module.

[0081] For example, in the embodiment of the present application, the proportional factor can be determined according to Formula 1:

[0082]

[0083] Among them, F scale1 Represents the scale factor corresponding to the first resolution normalization module, R inner1 Represents the resolution conversion parameter corresponding to the first resolution normalization module, R native Represents the dimension of the feature map input to the first resolution normalization module.

[0084] It should be noted that the resolution conversion parameter corresponding to the first resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter corresponding to the first resolution normalization module.

[0085] In an embodiment of the present application, the first resolution normalization module includes a positioning network, a grid generator and a sampler.

[0086] Optionally, in the embodiment of the present application, the feature map is input into the first resolution normalization module After that, the input feature map is passed to the localization network, which outputs the parameters of the affine transformation corresponding to the feature map. In the grid generator, a coordinate grid is generated by using the parameters of the affine transformation. The input feature map and the coordinate grid generated by the grid generator are used as inputs to the sampler. The module scale factor is normalized according to the first resolution, and the bilinear interpolation method is used to generate the reduced feature map.

[0087] It should be noted that Represents the scale plane R of the feature map with height H, width W, and number of channels C.

[0088] Among them, in the embodiment of the present application, based on adjusting the value of the resolution conversion parameter corresponding to the first resolution normalization module in the scale factor, the dimension of the feature map after dimensionality reduction can be adjusted accordingly.

[0089] Optionally, in order to achieve interpolation of the feature map output by the encoder, the embodiment of the present application may slightly adjust the scaling factor corresponding to the first resolution normalization module to ensure that the dimension of the output reduced-dimensional feature map is an integer.

[0090] For example, Figure 6 As shown, if the dimension of the feature map input to the first resolution normalization module is 4*4, based on the resolution conversion parameter corresponding to the first resolution normalization module, the feature map output by the encoder can be reduced in dimension to obtain the feature map after dimension reduction. For example, if the value of the resolution conversion parameter corresponding to the first resolution normalization module is 3, the dimension of the feature map after dimension reduction is 3*3.

[0091] Optionally, in the embodiment of the present application, the second resolution normalization module is used to perform dimension upscaling processing on the input feature map based on the resolution conversion parameter of the second resolution normalization module;

[0092] In the second resolution normalization module of the embodiment of the present application, a corresponding inverse proportional factor is determined according to a resolution conversion parameter of the second resolution normalization module, and based on the dimension of the feature map input to the second resolution normalization module and the inverse proportional factor corresponding to the second resolution normalization module, a dimensionality increase processing is performed on the feature map input to the second resolution normalization module.

[0093] For example, in the embodiment of the present application, the inverse proportional factor can be obtained according to Formula 2:

[0094]

[0095] in, Represents the inverse proportional factor corresponding to the second resolution normalization module, R inner2 Represents the resolution conversion parameter corresponding to the second resolution normalization module, R native Represents the dimension of the feature map of the output second resolution normalization module.

[0096] It should be noted that the resolution conversion parameter corresponding to the second resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter corresponding to the second resolution normalization module.

[0097] Optionally, the value of the resolution conversion parameter corresponding to the first resolution normalization module in the embodiment of the present application and the value of the resolution conversion parameter corresponding to the second resolution normalization module may be the same or different.

[0098] Optionally, in the embodiment of the present application, the feature map is input into the second resolution normalization module After that, the dimension of the feature map input to the second resolution normalization module is multiplied by the inverse proportional factor of the second resolution normalization module to obtain the feature map after dimensionality increase.

[0099] Optionally, in an embodiment of the present application, the dimension of the feature map input to the first resolution normalization module and the dimension of the feature map output by the second resolution normalization module are the same.

[0100] Among them, in the embodiment of the present application, based on adjusting the value of the resolution conversion parameter corresponding to the second resolution normalization module in the inverse proportional factor, the dimension of the feature map after dimensionality increase can be adjusted accordingly.

[0101] For example, Figure 7As shown, if the dimension of the feature map input to the second resolution normalization module is 3*3, then based on the resolution conversion parameter corresponding to the second resolution normalization module, the feature map output by the decoder can be upgraded to obtain the upgraded feature map. For example, if the dimension of the feature map input to the second resolution normalization module is 3*3, then the value of the resolution conversion parameter corresponding to the first resolution normalization module is 3, and a feature map with the same dimension as the feature map input to the first resolution normalization module is obtained. Figure 6 , the dimension of the feature map output by the second resolution normalization module is 4*4.

[0102] Among them, the feature map input to the second resolution normalization module in the embodiment of the present application is a feature map processed by the encoder or decoder between the first resolution normalization module and the second resolution normalization module.

[0103] Step S304: up-sample the feature map after the dimension conversion to obtain a target magnetic resonance image.

[0104] It should be noted that the embodiment of the present application can extract brain structures in magnetic resonance images, and can identify different brain regions of the extracted brain structures.

[0105] In the embodiment of the present application, the target magnetic resonance image includes a brain structure segmented from the magnetic resonance image, and the brain structure includes multiple brain regions.

[0106] Optionally, the present application provides several corresponding processing methods for the following situations:

[0107] Case 1: The decoder upsamples the feature map output by the first resolution normalization module.

[0108] In an embodiment of the present application, after the first resolution normalization module outputs the feature map, the feature map after dimensionality reduction is input into the decoder, and the decoder performs upsampling processing on the feature map after dimensionality reduction to obtain a target magnetic resonance image.

[0109] Case 2: The decoder upsamples the feature map output by the second resolution normalization module.

[0110] In an embodiment of the present application, after the second resolution normalization module outputs the feature map, the dimension-upgraded feature map is input into the decoder, and the decoder performs upsampling processing on the dimension-upgraded feature map to obtain a target magnetic resonance image.

[0111] Optionally, the input of the second resolution normalization module in the embodiment of the present application can be a feature map after dimensionality reduction processing by the first resolution normalization module, or can be a feature map after encoder feature extraction. Among them, the input of the encoder can be a feature map after dimensionality reduction processing by the first resolution normalization module.

[0112] Case 3: The decoder upsamples the feature map output by the encoder.

[0113] In an embodiment of the present application, after the first resolution normalization module outputs a feature map, the reduced-dimensional feature map is input into an encoder. After the encoder performs feature extraction on the reduced-dimensional feature map, the output reduced-dimensional feature map is input into a decoder. The decoder performs upsampling on the reduced-dimensional feature map to obtain a target magnetic resonance image.

[0114] Case 4: The decoder upsamples the feature map output by the previous decoder.

[0115] In an embodiment of the present application, the decoder upsamples the upsampled sample feature map output by the previous decoder to obtain a test label.

[0116] Optionally, the input of the previous decoder in the embodiment of the present application can be a feature map after dimensionality reduction processing by the first resolution normalization module, or a feature map after feature extraction by the encoder. Among them, the input of the encoder can be a feature map after dimensionality reduction processing by the first resolution normalization module.

[0117] Optionally, based on the above-mentioned trained segmentation model, the embodiment of the present application can also process the feature map through an adaptive attention mechanism in at least one encoder and / or at least one decoder in the trained segmentation model.

[0118] In an embodiment of the present application, the adaptive attention mechanism learns and activates the dynamic weighting process of each feature response generated by the convolution sequence in at least one encoder, and / or learns and activates the dynamic weighting process of each feature response generated by the convolution sequence in at least one decoder. Through this online weight calculation, nonlinear calculation processing outside the activation function is introduced into at least one encoder and / or at least one decoder. The segmentation accuracy of structures of different sizes in the image can be improved.

[0119] Optionally, in an embodiment of the present application, different brain regions in the target MRI image may be identified in a trained segmentation model according to the Desikan-Killiany-Tourville (DKT) atlas.

[0120] It should be noted that the brain structure in the embodiment of the present application may include 95 brain regions.

[0121] Based on the trained segmentation model, we can get Figure 8 The target MRI images corresponding to different planes are shown, where different brain regions can be distinguished by different colors.

[0122] Step S305: presenting the target MRI image in the display interface, and identifying different brain regions in the target MRI image.

[0123] An optional implementation is that the embodiment of the present application directly presents the target magnetic resonance image in the display interface.

[0124] Presented as Figure 8 Target MRI image shown.

[0125] Optionally, in the embodiment of the present application, a three-dimensional magnetic resonance image corresponding to the patient may also be determined based on the target magnetic resonance images corresponding to the patient in three planes respectively.

[0126] In an embodiment of the present application, magnetic resonance images corresponding to different planes are obtained by scanning the patient's brain, and target magnetic resonance images corresponding to different planes are determined by a trained segmentation model. Based on the target magnetic resonance images corresponding to the same plane, the three-dimensional magnetic resonance image corresponding to the patient can be determined.

[0127] Optionally, the present application embodiment provides a logic diagram for determining a three-dimensional nuclear magnetic resonance image. Fig. 9 As shown in the figure, the leftmost figure represents the patient's brain. After scanning the patient's brain, the MRI images corresponding to different planes are determined. The MRI images of the corresponding planes are processed by the trained segmentation model. For one MRI image, multiple segmented MRI images and the probability values ​​corresponding to each segmented MRI image are output. The segmented MRI image with the largest probability value is determined as the target MRI image through the softmax function. The target MRI images corresponding to multiple planes are layered and fused based on the preset weights to obtain the three-dimensional MRI image corresponding to the patient.

[0128] Optionally, another logical schematic diagram of determining a three-dimensional nuclear magnetic resonance image is provided in an embodiment of the present application. Fig.10 As shown, the leftmost figure represents the patient's brain. After scanning the patient's brain, the MRI images corresponding to different planes are determined, and the MRI images of the corresponding planes are processed by the trained segmentation model. The corresponding target MRI image is output for one plane, and the target MRI images corresponding to multiple planes are layered and fused to obtain the corresponding three-dimensional MRI image of the patient.

[0129] It should be noted that three-dimensional magnetic resonance imaging contains various brain regions.

[0130] Another optional implementation is to present a corresponding segmentation effect image after fusing the target nuclear magnetic resonance image and the nuclear magnetic resonance image.

[0131] In the present application embodiment, based on Figure 8 The target MRI image shown can be obtained as Fig.11 The corresponding segmentation renderings and the corresponding three-dimensional nuclear magnetic resonance images shown. In the embodiment of the present application, the segmentation renderings and the corresponding three-dimensional nuclear magnetic resonance images can be displayed to the user. Among them, the segmentation renderings and the three-dimensional nuclear magnetic resonance images include different brain regions marked, and different brain regions can be marked by different colors.

[0132] Optionally, the embodiment of the present application can obtain a corresponding segmentation effect diagram by inputting the nuclear magnetic resonance image and the target nuclear magnetic resonance image into the image viewing software. Fig.12 As shown, the embodiment of the present application provides a flowchart of training a segmentation model, and the specific steps are as follows:

[0133] Step S1201, input the training samples in the sample set into the initial segmentation model;

[0134] The training samples include sample images containing brain structures and sample labels corresponding to the sample images;

[0135] In the embodiment of the present application, since the MRI machine will collect MRI images of the patient's head in three directions when collecting MRI images of the patient's head, MRI images corresponding to three planes are obtained. Therefore, the training samples in the sample set of the embodiment of the present application will include sample images of brain structures corresponding to the three planes, and sample labels corresponding to each sample image.

[0136] It should be noted that, in the embodiment of the present application, the training samples in the sample set include sample images of a patient corresponding to three planes, and sample labels corresponding to the sample images of the three planes.

[0137] For example, the training samples may include sample images of the axial plane, the coronal plane, and the sagittal plane corresponding to the patient's brain.

[0138] Optionally, the sample label corresponding to the sample image in the embodiment of the present application is determined according to the following methods:

[0139] Method 1: Manually mark sample images to determine sample labels corresponding to the sample images.

[0140] Method 2: Determine the sample label corresponding to the sample image through the trained full convolutional neural network model;

[0141] Optionally, the number of encoder and / or decoder layers in the fully convolutional neural network model of the embodiment of the present application is greater than the number of encoder and / or decoder layers in the segmentation model.

[0142] In an embodiment of the present application, the number of codec layers of the fully convolutional neural network model is greater than the number of codec layers of the segmentation model.

[0143] Optionally, in an embodiment of the present application, the codec layer may include only an encoder, or only a decoder, or both an encoder and a decoder.

[0144] It should be noted that the fully convolutional neural network model has a larger number of encoder and / or decoder layers, which can improve the accuracy of the fully convolutional neural network model in determining the sample labels corresponding to the sample images.

[0145] Optionally, in an embodiment of the present application, sample images of different planes are input into trained fully convolutional neural network models corresponding to different planes.

[0146] It should be noted that the trained fully convolutional neural network models corresponding to different planes are obtained through multiple iterative training processes.

[0147] Optionally, in an embodiment of the present application, in each training process except the initial training process, the input samples of the full convolutional neural network model include initial samples and target samples, the sample labels corresponding to the initial samples are pre-configured, and the sample labels corresponding to the target samples are output by the full convolutional neural network model in the previous iterative training process.

[0148] It should be noted that the initial sample includes a sample image and a sample label corresponding to the initial sample, and the sample label corresponding to the initial sample can be pre-configured manually.

[0149] Among them, the multiple iterative training processes of the full convolutional neural network model can be divided into the initial training process and each training process other than the initial training process. These two processes are introduced as follows.

[0150] 1. The initial training process.

[0151] In an embodiment of the present application, during the initial training process of the fully convolutional neural network model, the initial samples are input into the fully convolutional neural network model.

[0152] Optionally, the embodiment of the present application may process the initial samples before the initial samples are input into the fully convolutional neural network model.

[0153] Exemplarily, the present application embodiment provides the following methods for processing the initial sample, as described below:

[0154] Method 1: Ghosting artifact enhancement processing.

[0155] Ghost noise is a phase-encoded motion that appears as a repetition of the scanned object in the image. Ghost noise is caused by periodic motion of tissue or fluid during the scan, affecting the data sampling in the phase-encoded direction.

[0156] Optionally, in an embodiment of the present application, for K-space data of a sample image corresponding to at least one initial sample, the K-space data of the sample image is modulated accordingly by a Fourier transform method to enhance the artifact.

[0157] It should be noted that each sample image has corresponding K-space data.

[0158] Exemplarily, for any sample image containing the brain, a corresponding random factor is set for the K-space data of the sample image through fast Fourier transform (FFT), and the data in the frequency encoding direction and the data in the phase encoding direction corresponding to the K-space data of the sample image are weighted respectively by the random factor.

[0159] It should be noted that if there are multiple repeated sample images of the patient's head, the same random factor can be used for the repeated sample images. For example, the random factor can be set to [0.85, 0.95], respectively for the frequency encoding direction and phase encoding direction corresponding to the K-space data. The number of repeated sample images of the patient's brain can be a random integer of [2, 4].

[0160] Method 2: Ringing artifact enhancement processing.

[0161] Ringing is a Gibbs phenomenon that occurs at boundaries with high contrast transitions. Ringing is caused by undersampling or truncation of high-frequency components in the image.

[0162] Optionally, the embodiment of the present application enhances the ringing artifacts of a sample image corresponding to any initial sample by applying a centralized fast Fourier transform to the brain spatial volume in three orthogonal directions and cutting the edges of the sample image along three axes according to preset cutting intervals.

[0163] Exemplarily, in the embodiment of the present application, the preset cutting interval may be [90, 120], and a random integer is determined in [90, 120], and the edge of the sample image is cut at the determined random integer.

[0164] Optionally, the embodiment of the present application cuts the edge of the sample image by processing the K-space data of the edge of the sample image.

[0165] Exemplarily, the embodiment of the present application processes the K-space data in a region not less than the random integer in [90, 120] to implement a process of cutting the edge of the sample image.

[0166] Method 3: elastic deformation processing.

[0167] Elastic deformation is a state-of-the-art method that learns shape-invariant representations by extending the training data through synthetically sensible transformations of the data.

[0168] Optionally, embodiments of the present application may increase our training data by applying a random elastic deformation algorithm.

[0169] In the embodiment of the present application, the elastic coefficient σ and the kernel size are controlled by setting the coefficient α to achieve image deformation.

[0170] Exemplarily, the present application embodiment takes any sample image as an example to introduce the application of the random elastic deformation algorithm as follows:

[0171] First, a random three-dimensional uniform displacement field is generated along each axis of the sample image. The obtained random three-dimensional uniform displacement field is smoothed using a Gaussian filter, and a value is randomly determined as the elastic coefficient σ according to a preset elasticity interval. The kernel size is 2[2σ]+1. For example, the preset elasticity interval can be [20, 30].

[0172] Then, according to the preset intensity interval, the coefficient α is randomly determined to scale them. The intensity of the deformation is controlled by the coefficient α. For example, the preset intensity interval may be [200, 500].

[0173] Finally, a reasonable deformed volume is obtained by interpolating the structured rectangular grid and the MRI volume.

[0174] Optionally, the size of the structured rectangular grid can be determined according to any integer value in [1, 256].

[0175] Optionally, in the embodiment of the present application, bilinear interpolation and bicubic interpolation are applied to the image and the label volume respectively.

[0176] In an embodiment of the present application, the processed initial samples are input into a fully convolutional neural network model.

[0177] Optionally, since manual configuration of sample labels corresponding to initial samples is inefficient and a large number of initial samples with sample labels cannot be obtained in a short time, in order to enhance the efficiency of obtaining labeled samples, an algorithm can be used to determine the sample labels corresponding to the unlabeled original images.

[0178] In the embodiment of the present application, the determined sample label corresponding to the original image may be an image.

[0179] It should be noted that the algorithm in the embodiment of the present application may be the algorithm in Freesurfer.

[0180] like Fig.13 As shown in FIG. 1 , a schematic diagram of sample input during the initial training process of an embodiment of the present application is shown. Fig.13 As shown, the initial samples with sample labels, the sample labels corresponding to the original images processed by Freesurfer, and the original images are input into the fully convolutional neural network model.

[0181] Among them, the sample labels corresponding to the original image and the sample labels corresponding to the initial samples can identify different brain regions in the brain structure.

[0182] Optionally, in an embodiment of the present application, the fully convolutional neural network model may include an encoder, a resolution normalization module, and a decoder. Steps A1-A3 are used to introduce the process in a round of the fully convolutional neural network model:

[0183] A1. Through the encoder in the fully convolutional neural network model, feature extraction is performed on the sample image corresponding to the initial sample and the original image to determine the corresponding sample feature map; wherein each sample feature map includes the image features of the initial sample or the image features of the original image.

[0184] A2. Through the resolution normalization module in the full convolutional neural network model, based on the resolution conversion parameters of the resolution normalization module, the sample feature map is dimensionally converted;

[0185] Optionally, the resolution normalization module in the fully convolutional neural network model of the embodiment of the present application includes a third resolution normalization module and a fourth resolution normalization module.

[0186] Among them, the input of the third resolution normalization module is connected to the encoder, the output of the fourth resolution normalization module is connected to the decoder, and at least one encoder and / or at least one decoder are included between the third resolution normalization module and the fourth resolution normalization module.

[0187] Optionally, in the embodiment of the present application, a third resolution normalization module is used to perform dimensionality reduction processing on the sample feature map output by the encoder based on the resolution conversion parameter of the third resolution normalization module;

[0188] In the third resolution normalization module of the embodiment of the present application, a corresponding scaling factor is determined according to a resolution conversion parameter of the third resolution normalization module, and based on the dimension of the sample feature map input into the third resolution normalization module and the scaling factor corresponding to the third resolution normalization module, a dimensionality reduction process is performed on the sample feature map input into the third resolution normalization module.

[0189] It should be noted that, in the embodiment of the present application, the scaling factor corresponding to the third resolution normalization module can be determined by referring to Formula 1, and the specific process will not be repeated here.

[0190] The resolution conversion parameter of the third resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter of the third resolution normalization module.

[0191] It should be noted that the third resolution normalization module in the embodiment of the present application may be completely the same as the first resolution normalization module, or may be different.

[0192] Optionally, in an embodiment of the present application, a sample feature map after dimensionality reduction is obtained by multiplying the dimension of the sample feature map input to the third resolution normalization module by a scaling factor of the third resolution normalization module.

[0193] Optionally, in the embodiment of the present application, the input sample feature map is subjected to dimensionality upscaling processing based on the resolution conversion parameters of the fourth resolution normalization module through the fourth resolution normalization module;

[0194] In the fourth resolution normalization module of the embodiment of the present application, a corresponding inverse proportional factor is determined according to the resolution conversion parameter of the fourth resolution normalization module, and based on the dimension of the sample feature map input to the fourth resolution normalization module and the inverse proportional factor corresponding to the fourth resolution normalization module, a dimensionality increase processing is performed on the sample feature map input to the fourth resolution normalization module.

[0195] It should be noted that, in the embodiment of the present application, the inverse proportional factor corresponding to the fourth resolution normalization module can be determined by referring to Formula 2, and the specific process will not be repeated here.

[0196] The resolution conversion parameter of the fourth resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter of the fourth resolution normalization module.

[0197] It should be noted that the fourth resolution normalization module in the embodiment of the present application may be completely the same as the second resolution normalization module, or may be different.

[0198] Optionally, in an embodiment of the present application, the dimension of the sample feature map input to the fourth resolution normalization module is multiplied by the fourth resolution normalization module scaling factor to obtain a sample feature map after dimensionality increase.

[0199] A3. Through the decoder in the fully convolutional neural network model, the sample feature map after dimension conversion is upsampled to obtain the test label.

[0200] It should be noted that the test label in the embodiment of the present application is an image.

[0201] Optionally, the present application provides several corresponding processing methods for the following situations:

[0202] Case 1: The decoder upsamples the sample feature map output by the third resolution normalization module.

[0203] In an embodiment of the present application, after the third resolution normalization module outputs the sample feature map, the reduced-dimensional sample feature map is input into the decoder, and the decoder upsamples the reduced-dimensional sample feature map to obtain a test label.

[0204] Case 2: The decoder upsamples the sample feature map output by the fourth resolution normalization module.

[0205] In an embodiment of the present application, after the fourth resolution normalization module outputs the sample feature map, the dimension-upgraded sample feature map is input into the decoder, and the decoder upsamples the dimension-upgraded sample feature map to obtain a test label.

[0206] Optionally, the input of the fourth resolution normalization module in the embodiment of the present application can be a sample feature map after dimensionality reduction processing by the third resolution normalization module, or a sample feature map after encoder feature extraction. Among them, the input of the encoder can be a sample feature map after dimensionality reduction processing by the third resolution normalization module.

[0207] Case 3: The decoder upsamples the sample feature map output by the encoder.

[0208] In an embodiment of the present application, after the third resolution normalization module outputs the sample feature map, the sample feature map after dimensionality reduction is input into the encoder. After the encoder performs feature extraction on the sample feature map after dimensionality reduction, the output sample feature map after dimensionality reduction is input into the decoder. The decoder performs upsampling on the sample feature map after dimensionality reduction to obtain a test label.

[0209] Case 4: The decoder upsamples the sample feature map output by the previous decoder.

[0210] In an embodiment of the present application, the decoder upsamples the upsampled sample feature map output by the previous decoder to obtain a test label.

[0211] Optionally, the input of the previous decoder in the embodiment of the present application can be a sample feature map after dimensionality reduction processing by the third resolution normalization module, or a sample feature map after encoder feature extraction. Among them, the input of the encoder can be a sample feature map after dimensionality reduction processing by the third resolution normalization module.

[0212] Optionally, based on the above-mentioned fully convolutional neural network model structure, an embodiment of the present application may add an adaptive attention mechanism in at least one encoder and / or at least one decoder in the fully convolutional neural network model.

[0213] In an embodiment of the present application, the adaptive attention mechanism learns and activates the dynamic weighting process of each feature response generated by the convolution sequence in at least one encoder, and / or learns and activates the dynamic weighting process of each feature response generated by the convolution sequence in at least one decoder. Through this online weight calculation, nonlinear calculation processing outside the activation function is introduced into at least one encoder and / or at least one decoder. The segmentation accuracy of structures of different sizes in the image can be improved.

[0214] Optionally, in an embodiment of the present application, a fully convolutional neural network model is trained using a k-fold interactive validation algorithm.

[0215] It should be noted that the embodiment of the present application may pre-set the k-fold interactive verification algorithm, for example, to a 5-fold interactive verification algorithm.

[0216] For example, if a 5-fold cross-validation algorithm is used to train a full convolutional neural network model, a cross-validation set can be formed based on the initial sample, the original image, and the sample label corresponding to the original image. The cross-validation set can include a test set and a training set.

[0217] Optionally, in an embodiment of the present application, the test set may include initial samples; the training set may include original images and sample labels corresponding to the original images, as well as initial samples.

[0218] It should be noted that the embodiment of the present application can allocate the determined multiple initial samples, and determine the initial samples in the test set and the initial samples in the training set according to a preset allocation rule.

[0219] Exemplarily, when determining the initial samples in the test set and the initial samples in the training set by using preset allocation rules, one-fifth of the initial samples in the total number of initial samples can be determined as the initial samples in the test set; and four-fifths of the initial samples in the total number of initial samples can be determined as the initial samples in the training set.

[0220] Based on the cross-validation set determined above, the full convolutional neural network model is trained according to the determined cross-validation set.

[0221] In an embodiment of the present application, for the fully convolutional neural network model corresponding to any plane, five fully convolutional neural network models can be trained through a five-fold interactive verification algorithm, and a fully convolutional neural network model corresponding to the plane can be obtained by screening the five fully convolutional neural network models.

[0222] Among them, in an embodiment of the present application, the Dice value corresponding to the full convolutional neural network model is determined based on a test label set corresponding to multiple sample images in a test set output by the full convolutional neural network model, and a sample label set corresponding to multiple sample images in the test set.

[0223] The embodiment of the present application screens five fully convolutional neural network models according to the numerical values ​​of the Dice values ​​corresponding to the five fully convolutional neural network models, and determines a fully convolutional neural network model corresponding to the plane.

[0224] In the embodiment of the present application, Dice is a set similarity measurement indicator, and the Dice value can be determined by Formula 3:

[0225]

[0226] Among them, Dice in Formula 3 represents the Dice value corresponding to the full convolutional neural network model; pred represents the test label set corresponding to multiple sample images in the test set output by the full convolutional neural network model; true represents the sample label set corresponding to multiple sample images in the test set.

[0227] pred∩true represents the intersection between the test label set corresponding to the multiple sample images and the sample label set corresponding to the multiple sample images; pred∪true represents the union between the test label set corresponding to the multiple sample images and the sample label set corresponding to the multiple sample images.

[0228] Exemplarily, when screening five fully convolutional neural network models, the fully convolutional neural network model corresponding to the largest Dice value can be determined as a fully convolutional neural network model corresponding to the plane based on the Dice value with the largest value among the Dice values ​​corresponding to different fully convolutional neural network models.

[0229] In an embodiment of the present application, based on a fully convolutional neural network model corresponding to the determined plane, a test label corresponding to the input sample image and a test label corresponding to the input original image can be obtained.

[0230] Among them, the embodiment of the present application determines the loss value corresponding to the plane based on the test label corresponding to the determined sample image, the test label corresponding to the determined original image, the sample label corresponding to the sample image, and the sample label corresponding to the original image.

[0231] Optionally, the embodiment of the present application obtains multiple label types corresponding to the sample images output by the initial fully convolutional neural network model;

[0232] It should be noted that the various label types corresponding to the sample images can be preset.

[0233] Optionally, an embodiment of the present application determines, for any label type, the probability value that each pixel point in the test label is the test label; determines the probability value of the test label being the label type based on the probability value that each pixel point in the test label is the label type; and determines the probability value that each pixel point in the sample image corresponding to the test label is the label type; determines the probability value that the sample label corresponding to the sample image is the label type based on the probability value that each pixel point in the sample image corresponding to the test label is the label type.

[0234] In the embodiment of the present application, based on the above process, the probability value of the test label corresponding to each label type and the probability value of the sample label corresponding to the sample image are determined, and the loss value between the test label and the sample label can be determined.

[0235] Optionally, an embodiment of the present application determines the loss value between the test label and the sample label based on preset weight parameters, probability values ​​corresponding to the test label for each label type, and probability values ​​corresponding to the sample label for each label type.

[0236] With respect to the above process, the embodiment of the present application determines the loss value between the test label and the sample label through Formula 4.

[0237] Optionally, if the plane is an axial plane, the loss value corresponding to any sample image in the plane for any label type can be determined by Formula 4, and the loss value corresponding to any original image in the plane for any label type can be determined by Formula 4. For ease of introduction, the sample image and the original image are referred to as images in the introduction to Formula 4.

[0238]

[0239] Among them, LA Indicates the loss value corresponding to any image in the axial plane when the label type is l; l,i represents the probability value corresponding to pixel i in the sample label corresponding to image x when the label type is l; p l,i (x) represents the probability value corresponding to pixel i in the test label corresponding to image x when the label type is l; l represents the type of label, and i represents the pixel point;

[0240] w i represents the weight parameter, wmedian freq. indicates the weight corresponding to the median frequency. The update times and update speed of the weight parameter are controlled by adjusting the median frequency. gradient Represents the gradient weight of the image, which is used to adjust the boundary in the image by adjusting the image gradient weight; w GM represents the weight corresponding to the outer gray matter mask in the image, and is used to adjust the pixels at the cortical boundary by adjusting the weight corresponding to the outer gray matter mask; Represents the weight corresponding to the deep groove mask in the image. By adjusting the weight corresponding to the deep groove mask, it is used to adjust the pixels at the thin white matter layer and the narrow groove position;

[0241] In the embodiment of the present application, by increasing w median freq. The weight corresponding to the median frequency can improve the classification ability of the model; by increasing w gradient Image gradient weights can refine the boundaries; by increasing w GM The weights corresponding to the outer gray matter mask in the image can emphasize the pixels at the cortical boundary; by increasing The weights corresponding to the deep groove masks in the image can emphasize pixels at thin white matter layers and narrow groove locations.

[0242] It should be noted that the white matter (WM) layer and deep groove masks in the image emphasize thin white matter layers and narrow grooves, defined by the voxels added by binary closing operations on the gray matter (GM) labels. GM ) emphasizes pixels at the cortical border, defined by voxels lost during brain mask erosion. In general, w GM and It aims to adjust the basic decision boundary to closely match the target segmentation at the location affected by the local volume effect, assign higher weights to narrow white matter layers, deep grooves and tissue boundaries, emphasize the boundaries of the cortex and cerebrospinal fluid (CSF), and is used to improve the segmentation quality of highly curved and complex areas of the cerebral cortex in images.

[0243] It should be noted that the sample image and the original image in the embodiment of the present application respectively correspond to at least one label type, and through the fully convolutional neural network model, it can also be obtained that the sample image and the original image respectively correspond to at least one test label category.

[0244] In the embodiment of the present application, at least one label type corresponding to the sample image and at least one corresponding test label category obtained by the model may be completely the same or different. The present application does not impose any limitation on this.

[0245] Based on Formula 4, we can obtain the loss value corresponding to any label category for any sample image in the axial plane and the loss value corresponding to any label category for any original image in the axial plane.

[0246] It should be noted that, for the sake of ease of introduction, the label type corresponding to the sample image and the test label category corresponding to the sample image are collectively referred to as label category.

[0247] In the embodiment of the present application, in formula 4, for the fully convolutional neural network model corresponding to any plane, for any label category, the loss value of the sample image corresponding to the test label category is determined based on the probability value of the pixel point in any sample image corresponding to the label type and the probability value of the pixel point in the sample label corresponding to the sample image corresponding to the test label category.

[0248] Optionally, in an embodiment of the present application, for the fully convolutional neural network model corresponding to any plane, for any label category, the loss value corresponding to any label category of the plane is determined by performing calculations on the loss values ​​corresponding to each sample image.

[0249] Exemplarily, in the embodiment of the present application, the loss values ​​corresponding to each sample image can be processed by weighted average operation to determine the loss value corresponding to any label category of the plane.

[0250] Optionally, in the embodiment of the present application, for any label category, a loss value corresponding to any label category of the plane can be determined by performing a weighted average operation on each sample image.

[0251] For example, if the fully convolutional neural network model corresponding to the plane inputs sample image 1, original image 1 and original image 2, sample image 1 corresponds to label category 1 with a loss value of 0.4 and label category 2 with a loss value of 0.5; original image 1 corresponds to label category 1 with a loss value of 0.6 and label category 2 with a loss value of 0.4; original image 2 corresponds to label category 1 with a loss value of 0.5 and label category 2 with a loss value of 0.4. For label category 1, the weighted average operation of the values ​​0.4, 0.6 and 0.5 corresponding to label category 1 can obtain the loss value corresponding to label category 1 as 0.5. Similarly, the loss value corresponding to label category 2 is determined according to the same method, which will not be repeated here.

[0252] For the fully convolutional neural network model corresponding to any plane, the loss value corresponding to at least one label category can be obtained.

[0253] Optionally, in the embodiment of the present application, for the fully convolutional neural network model corresponding to other planes, the loss value L corresponding to any sample image corresponding to the coronal plane for any label type can be obtained by formula 4: C , and the loss value L corresponding to any original image of the coronal plane for any label type C .

[0254] Similarly, in the embodiment of the present application, the loss value L corresponding to any sample image in the sagittal plane for any label type can also be obtained. S , and the loss value L corresponding to any original image of the sagittal plane for any label type S .

[0255] It should be noted that any sample image in the coronal plane corresponds to at least one loss value L C , and any original image in the coronal plane corresponds to at least one loss value L C ; Any sample image in the sagittal plane corresponds to at least one loss value L S , and any original image in the sagittal plane corresponds to at least one loss value L S .

[0256] Based on the same principle as obtaining the loss value corresponding to at least one label category for the coronal plane, the loss values ​​corresponding to at least one label category corresponding to the coronal plane and the sagittal plane respectively can be obtained.

[0257] It should be noted that, in the embodiment of the present application, the total loss value corresponding to a plane includes the total loss value corresponding to at least one label category.

[0258] For example, the axial plane may correspond to the label category 1 with a loss value of 0.4 and the label category 2 with a loss value of 0.5; the coronal plane may correspond to the label category 1 with a loss value of 0.6 and the label category 2 with a loss value of 0.6; the sagittal plane may correspond to the label category 1 with a loss value of 0.5 and the label category 2 with a loss value of 0.4. Then the total loss value corresponding to the three planes is the label category 1 with a loss value of 0.5 and the label category 2 with a loss value of 0.5.

[0259] Optionally, in an embodiment of the present application, a total loss value corresponding to the multiple planes is determined by performing calculations on the loss values ​​of at least one label category corresponding to the multiple planes respectively.

[0260] Exemplarily, for any label category, the loss values ​​corresponding to the label category of multiple planes are used to determine the total loss value corresponding to the label category of multiple planes through formula five.

[0261]

[0262] Among them, L total Represents the total loss value of multiple planes corresponding to the label category; L A (m) represents the loss value corresponding to the pixel m in the sample image of the axial plane when it is of the label category; L C (m) represents the loss value corresponding to the pixel m in the sample image of the coronal plane when it is of the label category; L S (m) represents the loss value corresponding to pixel point m in the sample image of the sagittal plane for the label category; pixel point m is selected from multiple pixel points in the sample image, and when m=1, it is determined to be the pixel point corresponding to sequence number 1; M represents the total number of available pixel points in the sample image.

[0263] Optionally, in the embodiment of the present application, the available pixel points in the sample image may be all the pixel points in the sample image.

[0264] Among them, in the embodiment of the present application, the intersection of three planes is used to reconstruct the three-dimensional structure of the brain structure of the image. D() represents the distance between the pixel points on the intersection line of each two planes. Then D(AC, CA) represents the distance between the pixel points on the intersection line between the sample image of the axial plane and the corresponding sample image of the coronal plane; D(CS, SC) represents the distance between the pixel points on the intersection line between the sample image of the coronal plane and the corresponding sample image of the sagittal plane; D(AS, SA) represents the distance between the pixel points on the intersection line between the sample image of the axial plane and the corresponding sample image of the sagittal plane.

[0265] Optionally, the embodiment of the present application may measure the distance between the pixel points on the intersection line of every two planes by using the average KLD (Kullback-Leibler) method.

[0266] Optionally, the embodiment of the present application can adjust the full convolutional neural network model corresponding to the three planes based on the total loss value corresponding to at least one label category corresponding to the three planes.

[0267] Among them, the embodiment of the present application can adjust the gradient of the fully convolutional neural network model corresponding to each plane respectively through the total loss value.

[0268] 2. Each training process except the initial training process.

[0269] In an embodiment of the present application, in each initial sample training process except the initial training process, the input samples of the full convolutional neural network model include initial samples and target samples.

[0270] Optionally, the initial sample and the target sample may be processed before being input into the fully convolutional neural network model.

[0271] In the embodiments of the present application, the way of processing the initial sample and the target sample can refer to the way of processing the initial sample during the initial training of the full convolutional neural network model, and the present application will not go into details here.

[0272] Among them, the sample labels corresponding to the initial samples are pre-configured, and the sample labels corresponding to the target samples are the output of the fully convolutional neural network model in the last iterative training process.

[0273] Optionally, for a target sample, a flowchart of determining a target sample is provided in an embodiment of the present application, by: Fig.14 The flowchart shown determines the target sample input during each iterative training process. The specific steps are as follows:

[0274] Step S1401, obtaining a candidate sample set;

[0275] The candidate sample set includes at least one candidate sample that is not configured with a sample label.

[0276] Step S1402: for any candidate sample, determine the sample label output by the full convolutional neural network model of the candidate sample during the previous multiple iterative training processes; determine the uncertainty parameter of the sample label corresponding to the candidate sample according to the sample label output by the full convolutional neural network model during the previous multiple iterative training processes;

[0277] Among them, the value of the uncertainty parameter is negatively correlated with the number of identical sample labels in the sample labels output by the full convolutional neural network model during the previous multiple iterative training processes;

[0278] Exemplarily, if the current training is the second iteration, the candidate sample is input into the fully convolutional neural network model obtained after the initial training to obtain the sample label corresponding to the candidate sample.

[0279] It should be noted that in the embodiment of the present application, by inputting a candidate sample into the fully convolutional neural network model, a sample label corresponding to the candidate sample will be obtained.

[0280] Optionally, in the embodiment of the present application, the sample label corresponding to the candidate sample can be determined in the following ways:

[0281] Method 1: After inputting the candidate sample corresponding to a certain plane, a fully convolutional neural network model corresponding to the plane can output the test label corresponding to at least one test label category corresponding to the candidate sample, as well as the probability values ​​corresponding to each test label category. The test label corresponding to the probability value with the largest probability value is determined by the Softmax function, and the test label is used as the sample label corresponding to the candidate sample.

[0282] Method 2: After inputting the candidate sample corresponding to a plane, the sample label corresponding to the candidate sample can be directly determined through a fully convolutional neural network model corresponding to the plane.

[0283] Optionally, the uncertainty parameter of the sample label corresponding to the candidate sample is determined in the following ways in the embodiment of the present application, as described below:

[0284] Method 1: In the embodiment of the present application, the uncertainty parameter of the sample label corresponding to the candidate sample can be obtained by formula 6, as described below:

[0285]

[0286] Among them, u represents the uncertainty parameter, K represents the total number of iterations, and y i represents the sample label generated by iteration i. SUM(y i ≠y i-1 ) represents y i With y i-1 Whether the preset threshold is met, if the preset threshold is met, it is 1, SUM(y i >0) indicates y i exist.

[0287] It should be noted that in the embodiment of the present application, the model outputs the sample label corresponding to the initial sample, but the sample label may not necessarily exist. It may be that due to the large amount of data in the model, the sample label is wrong. Therefore, it is necessary to determine whether the sample label exists.

[0288] Among them, the sample label of the embodiment of the present application is an image.

[0289] Optionally, in the embodiment of the present application, by determining y i With y i-1 The similarity between them is used to judge y i With y i-1 Whether the preset threshold is met.

[0290] In the present application embodiment, determine y i With y i-1 If the similarity between them is less than a preset threshold, it is determined that the preset threshold is met.

[0291] Optionally, the embodiment of the present application may determine y by an IOU algorithm. i With y i-1 If y i With y i-1 The similarity between them is 0.95, and the preset threshold is 0.9, then it is determined that the preset threshold is not met.

[0292] Method 2: According to the sample labels corresponding to the candidate samples and the sample labels output in the previous multiple iterations of training, it is judged whether the preset conditions are met. If the preset conditions are met, it is determined that there is uncertainty.

[0293] In the embodiment of the present application, after determining that the candidate sample has uncertainty, the corresponding uncertainty parameter is determined.

[0294] Exemplarily, if it is determined that the candidate sample has uncertainty, a corresponding uncertainty parameter is determined. For example, the corresponding uncertainty parameter may be 1.

[0295] Optionally, the preset condition in the embodiment of the present application may be that the similarity between the sample label corresponding to the candidate sample and the sample label output in the previous multiple iterations of training does not meet the similarity threshold.

[0296] Optionally, the embodiment of the present application can determine the similarity between sample labels by an IOU algorithm, or refer to Formula 3 to determine that the Dice value between sample labels represents the similarity between sample labels.

[0297] In the embodiment of the present application, if the similarity threshold is a numerical value, then if it is less than the numerical value, it is determined that the similarity threshold is not met; if the similarity threshold is a range, then if it exceeds the range, it is determined that the similarity threshold is not met.

[0298] For example, taking the similarity threshold of 0.95 as an example, if the similarity between sample labels is 0.8, it is determined that the similarity threshold is not met; taking the similarity threshold of [0.95, 1] ​​as an example, if the similarity between sample labels is 0.8, it is determined that the similarity threshold is not met.

[0299] Optionally, in an embodiment of the present application, if the similarity between multiple sample labels is determined, a target similarity can be determined by performing a weighted average operation on the similarities between the multiple sample labels, and whether the sample label corresponding to the candidate sample satisfies the similarity threshold can be determined based on the target similarity; it is also possible to judge whether the similarity between each sample label satisfies the similarity threshold. If the probability of not satisfying the similarity threshold exceeds a preset threshold, it is determined that the sample label corresponding to the candidate sample does not satisfy the similarity threshold.

[0300] Step S1403: Filter target samples from the candidate sample set according to the uncertainty parameters of the sample labels corresponding to the candidate samples.

[0301] Optionally, in the embodiment of the present application, target samples are screened from the candidate sample set based on uncertainty parameters of sample labels corresponding to each candidate sample and a preset screening threshold.

[0302] In the embodiment of the present application, if it is not less than the screening threshold, the corresponding candidate sample is used as the target sample.

[0303] Exemplarily, taking the screening threshold of 0.9 as an example, if the uncertainty parameter is 0.9, the corresponding candidate sample is used as the target sample.

[0304] Optionally, the embodiment of the present application may process the initial sample and the target sample with the sample label before inputting the initial sample and the target sample into the full convolutional neural network model.

[0305] For the above process of processing the initial sample and the target sample, the process of processing the initial sample illustrated in the above initial training process can be referred to. The initial sample and the target sample with the sample label are processed based on the same method. This application will not be repeated here.

[0306] like Fig.15 As shown in FIG. 1 , a schematic diagram of sample input in each training process other than the initial training process in an embodiment of the present application. Fig.15 As shown, the initial sample and the target sample with sample label are input into the full convolutional neural network model. Among them, the target sample with sample label is obtained by screening the candidate samples with sample label, and the sample label corresponding to the candidate sample is output by the full convolutional neural network model according to the input candidate sample.

[0307] It should be noted that the sample labels corresponding to the candidate samples and the sample labels corresponding to the target samples can identify different brain regions in the brain structure.

[0308] In an embodiment of the present application, the initial sample and the target sample with the sample label are input into the fully convolutional neural network model, and the initial sample and the target sample with the sample label are processed by the fully convolutional neural network model.

[0309] Optionally, in an embodiment of the present application, the fully convolutional neural network model may include an encoder, a resolution normalization module, and a decoder. Steps B1-B3 are used to introduce the process in a round of the fully convolutional neural network model:

[0310] B1. Through the encoder in the fully convolutional neural network model, feature extraction is performed on the sample image corresponding to the initial sample and the target sample to determine the corresponding sample feature map; wherein each sample feature map includes the image features of the sample image corresponding to the initial sample or the image features of the target sample.

[0311] B2. Through the resolution normalization module in the full convolutional neural network model, based on the resolution conversion parameters of the resolution normalization module, the sample feature map is dimensionally converted;

[0312] Optionally, the resolution normalization module in the fully convolutional neural network model of the embodiment of the present application includes a third resolution normalization module and a fourth resolution normalization module.

[0313] Optionally, in the embodiment of the present application, a third resolution normalization module is used to perform dimensionality reduction processing on the sample feature map output by the encoder based on the resolution conversion parameter of the third resolution normalization module;

[0314] In the third resolution normalization module of the embodiment of the present application, a corresponding scaling factor is determined according to a resolution conversion parameter of the third resolution normalization module, and based on the dimension of the sample feature map input into the third resolution normalization module and the scaling factor corresponding to the third resolution normalization module, a dimensionality reduction process is performed on the sample feature map input into the third resolution normalization module.

[0315] It should be noted that, in the embodiment of the present application, the scaling factor corresponding to the third resolution normalization module can be determined by referring to Formula 1, and the specific process will not be repeated here.

[0316] The resolution conversion parameter of the third resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter of the third resolution normalization module.

[0317] It should be noted that the third resolution normalization module in the embodiment of the present application may be completely the same as the first resolution normalization module, or may be different.

[0318] Optionally, in an embodiment of the present application, a sample feature map after dimensionality reduction is obtained by multiplying the dimension of the sample feature map input to the third resolution normalization module by a scaling factor of the third resolution normalization module.

[0319] Optionally, in the embodiment of the present application, the input sample feature map is subjected to dimensionality upscaling processing based on the resolution conversion parameters of the fourth resolution normalization module through the fourth resolution normalization module;

[0320] In the fourth resolution normalization module of the embodiment of the present application, a corresponding inverse proportional factor is determined according to the resolution conversion parameter of the fourth resolution normalization module, and based on the dimension of the sample feature map input to the fourth resolution normalization module and the inverse proportional factor corresponding to the fourth resolution normalization module, a dimensionality increase processing is performed on the sample feature map input to the fourth resolution normalization module.

[0321] It should be noted that, in the embodiment of the present application, the inverse proportional factor corresponding to the fourth resolution normalization module can be determined by referring to Formula 2, and the specific process will not be repeated here.

[0322] The resolution conversion parameter of the fourth resolution normalization module in the embodiment of the present application may be an adjustable value. The embodiment of the present application is tested through a large amount of data to determine a suitable value as the resolution conversion parameter of the fourth resolution normalization module.

[0323] It should be noted that the fourth resolution normalization module in the embodiment of the present application may be completely the same as the second resolution normalization module, or may be different.

[0324] Optionally, in an embodiment of the present application, the dimension of the sample feature map input to the fourth resolution normalization module is multiplied by the fourth resolution normalization module scaling factor to obtain a sample feature map after dimensionality increase.

[0325] B3. Through the decoder in the fully convolutional neural network model, the sample feature map after dimension conversion is upsampled to obtain the test label.

[0326] It should be noted that the test label in the embodiment of the present application is an image.

[0327] Optionally, several situations illustrated in step A3 may also occur in step B3, and these situations are processed by referring to the processing method in step A3. This application will not repeat them here.

[0328] Optionally, based on the above-mentioned fully convolutional neural network model structure, the embodiment of the present application can add an adaptive attention mechanism to at least one encoder and / or at least one decoder in the fully convolutional neural network model. For a detailed introduction, please refer to the introduction in the initial training process, which will not be repeated here.

[0329] Optionally, in an embodiment of the present application, a fully convolutional neural network model is trained using a k-fold interactive validation algorithm.

[0330] For example, if a 5-fold cross-validation algorithm is used to train a full convolutional neural network model, a cross-validation set can be formed based on the initial sample, the target sample, and the sample label corresponding to the target sample. The cross-validation set can include a test set and a training set.

[0331] Optionally, in an embodiment of the present application, the test set may include an initial sample; the training set may include a target sample and a sample label corresponding to the target sample, as well as the initial sample.

[0332] It should be noted that the embodiment of the present application can allocate multiple initial samples and determine the initial samples in the test set and the initial samples in the training set according to a preset allocation rule.

[0333] Exemplarily, when determining the initial samples in the test set and the initial samples in the training set by using preset allocation rules, one-fifth of the initial samples in the total number of initial samples can be determined as the initial samples in the test set; and four-fifths of the initial samples in the total number of initial samples can be determined as the initial samples in the training set.

[0334] Based on the cross-validation set determined above, the full convolutional neural network model is trained according to the determined cross-validation set.

[0335] In the embodiment of the present application, the Dice value corresponding to the full convolutional neural network model is determined based on the test label set corresponding to multiple sample images in the test set output by the full convolutional neural network model, and the sample label set corresponding to multiple sample images in the test set. For the specific process, refer to Formula 3.

[0336] The embodiment of the present application screens five fully convolutional neural network models according to the numerical values ​​of the Dice values ​​corresponding to the five fully convolutional neural network models, and determines a fully convolutional neural network model corresponding to each plane.

[0337] Optionally, the embodiment of the present application determines the total loss value corresponding to multiple planes according to the loss value corresponding to each plane, and adjusts the full convolutional neural network model according to the total loss value. The specific process refers to the introduction of the initial training process, which will not be repeated here.

[0338] Based on the above adjustment of the training process of the full convolutional neural network model, when the loss values ​​of the full convolutional neural network models corresponding to the multiple planes respectively meet the preset threshold, the training process of the full convolutional neural network model is completed. The embodiment of the present application determines the sample label corresponding to the sample image through the trained full convolutional neural network model.

[0339] Optionally, in the embodiment of the present application, the segmentation model in the embodiment of the present application can be trained according to the determined sample labels.

[0340] It should be noted that the trained full convolutional neural network model in the embodiment of the present application can determine high-precision sample labels, and compared with the process of manually determining sample labels, sample labels can be obtained more efficiently. Compared with the segmentation model trained based on fewer sample labels and low-precision sample labels, the segmentation model in the embodiment of the present application is trained based on a large number of high-precision sample labels, which can improve the segmentation accuracy of the trained segmentation model.

[0341] Based on the above process, the embodiment of the present application obtains a sample set that is input into the initial segmentation model.

[0342] In the embodiment of the present application, for the initial segmentation model corresponding to any plane obtained, after the training samples in the sample set are input into the initial segmentation model of the corresponding plane, the training samples in the sample set are processed as follows in the initial segmentation model:

[0343] Step S1202: extract features from the sample image based on the initial segmentation model to determine a sample feature map including image features of the sample image;

[0344] Optionally, feature extraction is performed on the sample image through an encoder in the initial segmentation model to determine a sample feature map.

[0345] Step S1203: Perform dimension conversion processing on the sample feature map according to the resolution conversion parameters corresponding to the initial segmentation model;

[0346] Optionally, a resolution normalization module in the initial segmentation model is used to perform dimension conversion on the sample feature map based on a resolution conversion parameter of the resolution normalization module.

[0347] Step S1204, performing up-sampling processing on the sample feature map after the dimension conversion processing to obtain a nuclear magnetic resonance reference image;

[0348] Optionally, the sample feature map after the dimension conversion is upsampled through a decoder in the initial segmentation model to obtain a nuclear magnetic resonance reference image.

[0349] The MRI reference image includes a brain structure segmented from the sample image, and the brain structure includes a plurality of brain regions.

[0350] Optionally, several situations illustrated in step S304 may also occur in step S1204, and these situations are processed by referring to the processing method in S304. This application will not repeat them here.

[0351] Optionally, based on the above-mentioned fully convolutional neural network model structure, the embodiment of the present application can add an adaptive attention mechanism to at least one encoder and / or at least one decoder in the initial segmentation model. For a specific introduction, please refer to the introduction in step S304, which will not be repeated here.

[0352] Based on the above steps S1202 to S1204, the processing of the training samples in the initial segmentation model can be completed.

[0353] Optionally, the embodiment of the present application may train the initial segmentation model through a k-fold interactive validation algorithm.

[0354] Exemplarily, if the 5-fold cross-validation algorithm is used to train the initial segmentation model, the sample labels corresponding to the multiple training samples in the sample set include sample labels determined by manual labeling and sample labels determined by the trained full convolutional neural network model. The cross-validation set corresponding to the initial segmentation model is composed of the sample images and the corresponding sample labels determined by manual labeling, and the sample images and the corresponding sample labels determined by the trained full convolutional neural network model.

[0355] Optionally, in an embodiment of the present application, the sample image and the corresponding sample labels determined by manual labeling may be the same as the initial samples input into the fully convolutional neural network model.

[0356] The cross-validation set corresponding to the initial segmentation model may include a test set and a training set.

[0357] Optionally, in an embodiment of the present application, the test set may include sample images and corresponding sample labels determined by manual labeling; the training set may include sample images and corresponding sample labels determined by manual labeling, as well as sample images and corresponding sample labels determined by a trained fully convolutional neural network model.

[0358] It should be noted that the embodiments of the present application can allocate multiple determined sample images and corresponding sample labels determined by manual marking, and determine the sample images in the test set and the corresponding sample labels determined by manual marking, as well as the sample images in the training set and the corresponding sample labels determined by manual marking according to preset allocation rules.

[0359] Exemplarily, when the sample images in the test set and the corresponding sample labels determined by manual marking, as well as the sample images in the training set and the corresponding sample labels determined by manual marking are determined by preset allocation rules, one-fifth of the sample images in the total number of multiple sample images corresponding to the multiple sample labels determined by manual marking, and the manually marked sample labels corresponding to the one-fifth of the sample images, can be determined as the sample images in the test set and the corresponding sample labels determined by manual marking; and four-fifths of the sample images in the total number of multiple sample images corresponding to the multiple sample labels determined by manual marking, and the manually marked sample labels corresponding to the four-fifths of the sample images, can be determined as the sample images in the training set and the corresponding sample labels determined by manual marking.

[0360] Based on the cross-validation set corresponding to the initial segmentation model determined above, the initial segmentation model is trained according to the cross-validation set corresponding to the determined initial segmentation model.

[0361] In the embodiment of the present application, the specific processing process of the 5-fold interactive verification algorithm refers to the specific introduction in the initial training process. No further description is given here. Finally, a segmentation model corresponding to each of the multiple planes is obtained by screening.

[0362] The embodiment of the present application determines the loss value corresponding to the plane according to the segmentation model, and adjusts the model parameters of the initial segmentation model according to the determined loss value, which is specifically introduced in the following steps.

[0363] Step S1205: determine a loss value according to the MRI reference image and the sample label, and adjust the model parameters of the initial segmentation model according to the determined loss value until the determined loss value is within a preset range.

[0364] The model parameters of the initial segmentation model include resolution conversion parameters.

[0365] Optionally, the embodiment of the present application determines the loss value according to the NMR reference image and the sample label, as described in detail in steps C1-C3:

[0366] C1. Obtain multiple label types corresponding to the MRI reference image output by the initial segmentation model;

[0367] C2. For any label type, determine the probability value of each pixel in the NMR reference image being the label type; determine the probability value of the NMR reference image being the label type based on the probability value of each pixel in the NMR reference image being the label type; and determine the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type; determine the probability value of the sample label corresponding to the sample image being the label type based on the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type;

[0368] C3. Determine the loss value between the NMR reference image and the sample label according to the preset weight parameters, the probability value corresponding to each label type of the NMR reference image, and the probability value corresponding to each label type of the sample label.

[0369] Based on the above process, the embodiment of the present application can refer to Formula 4 and the process introduction of using Formula 4 to determine the loss values ​​corresponding to different planes based on the nuclear magnetic resonance reference image and the sample label.

[0370] Optionally, the embodiment of the present application determines the total loss values ​​corresponding to the multiple planes according to the determined loss values ​​corresponding to the different planes, referring to Formula 5 and the process introduction of using Formula 5. The present application will not repeat them here.

[0371] The embodiment of the present application adjusts the model parameters of the initial segmentation model according to the total loss values ​​corresponding to multiple planes until the determined loss value is within a preset range, thereby obtaining a trained segmentation model.

[0372] like Fig.16 As shown, a schematic diagram of the overall process of a segmentation model training process in an embodiment of the present application. Fig.16As shown, in the training process of the full convolutional neural network model, for the initial training process of the full convolutional neural network model, the initial sample with sample label, the original image and the corresponding sample label are input into the full convolutional neural network model, and the full convolutional neural network model is trained for the first time, wherein the initial sample with sample label can be image processed before being input into the full convolutional neural network model. Based on the full convolutional neural network model after the initial training, the input candidate sample is segmented, and the candidate sample and the corresponding sample label are output, and the target sample and the corresponding sample label are obtained by screening. The initial sample with sample label, the target sample and the corresponding sample label are image processed, and input into the full convolutional neural network model for the second training. Based on the full convolutional neural network model after the second training, the input candidate sample is segmented, and the specific process refers to the second training process, which will not be repeated here. The full convolutional neural network model is iteratively trained multiple times through the above process. Based on the initial sample with label, the target sample and the corresponding sample label output by the trained full convolutional neural network model, the segmentation model is iteratively trained.

[0373] It should be noted that the sample labels can identify different brain regions in the brain structure, wherein different brain regions can be identified by different colors.

[0374] Optionally, the embodiment of the present application can also screen the candidate samples for the next round of input of the full convolutional neural network model based on the determined target samples and the corresponding sample labels.

[0375] Among them, in the embodiment of the present application, the candidate samples corresponding to the screened target samples can be used as candidate samples for input into the next round of the fully convolutional neural network model.

[0376] It should be noted that in the embodiments of the present application, the full convolutional neural network model can be referred to as Big Brain-VNNet, and the segmentation model can be referred to as Small Brain-VNNet.

[0377] Optionally, the embodiments of the present application may add an adaptive attention mechanism in the encoder of the fully convolutional neural network model, and add an adaptive attention mechanism in the decoder of the segmentation model.

[0378] like Fig.17 As shown, an image segmentation device 1700 for a nuclear magnetic resonance image in an embodiment of the present application includes an acquisition module 1701 , a segmentation module 1702 , and a presentation module 1703 .

[0379] An acquisition module 1701 is used to acquire a nuclear magnetic resonance image of a human head and input the nuclear magnetic resonance image into a trained segmentation model;

[0380] The segmentation module 1702 is used to extract features from the nuclear magnetic resonance image based on the trained segmentation model, and determine a feature map including image features of the nuclear magnetic resonance image; perform dimension conversion processing on the feature map according to the resolution conversion parameter determined in the training process of the trained segmentation model; perform upsampling processing on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions;

[0381] The presentation module 1703 is used to present the target MRI image in a display interface and identify different brain regions in the target MRI image.

[0382] In a possible embodiment, the resolution conversion parameters include resolution conversion parameters corresponding to the first resolution normalization module in the trained segmentation model and resolution conversion parameters corresponding to the second resolution normalization module in the trained segmentation model;

[0383] The segmentation module 1702 is specifically used for:

[0384] Performing dimensionality reduction processing on the feature map according to a resolution conversion parameter corresponding to the first resolution normalization module;

[0385] According to the resolution conversion parameter corresponding to the second resolution normalization module, the feature map after the dimensionality reduction processing is increased in dimension.

[0386] In a possible embodiment, the segmentation model is trained in the following manner:

[0387] Inputting training samples in the sample set into an initial segmentation model; the training samples include sample images containing a human head and sample labels corresponding to the sample images;

[0388] Based on the initial segmentation model, feature extraction is performed on the sample image to determine a sample feature map including image features of the sample image; according to a resolution conversion parameter corresponding to the initial segmentation model, dimension conversion is performed on the sample feature map; upsampling is performed on the sample feature map after the dimension conversion to obtain a nuclear magnetic resonance reference image; the nuclear magnetic resonance reference image includes a brain structure segmented from the sample image, and the brain structure includes a plurality of brain regions;

[0389] A loss value is determined according to the nuclear magnetic resonance reference image and the sample label, and model parameters of the initial segmentation model are adjusted according to the determined loss value until the determined loss value is within a preset range; wherein the model parameters of the initial segmentation model include resolution conversion parameters.

[0390] In a possible embodiment, the sample label corresponding to the sample image is determined according to the following method:

[0391] Inputting the sample image into a trained fully convolutional neural network model;

[0392] Determine the sample label corresponding to the sample image through a trained full convolutional neural network model; wherein the number of encoder and / or decoder layers in the full convolutional neural network model is greater than the number of encoder and / or decoder layers in the segmentation model.

[0393] In a possible embodiment, the trained sample segmentation model is obtained through multiple iterative training processes;

[0394] Among them, in each initial sample training process except the initial training process, the input samples of the full convolutional neural network model include initial samples and target samples, the sample labels corresponding to the initial samples are pre-configured, and the sample labels corresponding to the target samples are output by the full convolutional neural network model in the last iterative training process.

[0395] In a possible embodiment, the target sample input during each iterative training process is determined according to the following method:

[0396] Acquire a candidate sample set, wherein the candidate sample set includes at least one candidate sample that is not configured with a sample label;

[0397] For any candidate sample, determine the sample label output by the full convolutional neural network model of the candidate sample during the previous multiple iterative training processes; determine the uncertainty parameter of the sample label corresponding to the candidate sample according to the sample label output by the full convolutional neural network model during the previous multiple iterative training processes; wherein the value of the uncertainty parameter is negatively correlated with the number of identical sample labels in the sample labels output by the full convolutional neural network model during the previous multiple iterative training processes;

[0398] According to the uncertainty parameters of the sample labels corresponding to the candidate samples, the target samples are screened from the candidate sample set.

[0399] In a possible embodiment, the image processing device is further used for:

[0400] Acquire multiple label types corresponding to the nuclear magnetic resonance reference image output by the initial segmentation model;

[0401] For any label type, determine the probability value of each pixel in the NMR reference image being the label type; determine the probability value of the NMR reference image being the label type based on the probability value of each pixel in the NMR reference image being the label type; and determine the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type; determine the probability value of the sample label corresponding to the sample image being the label type based on the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type;

[0402] The loss value between the NMR reference image and the sample label is determined according to the preset weight parameters, the probability value corresponding to each label type of the NMR reference image, and the probability value corresponding to each label type of the sample label.

[0403] Based on the same inventive concept as the above-mentioned method embodiment, an electronic device is also provided in the embodiment of the present application. The principle of solving the problem by the electronic device is similar to the method of the above-mentioned embodiment. Therefore, the implementation of the electronic device can refer to the implementation of the above-mentioned method, and the repeated parts will not be repeated.

[0404] See also Fig.18 As shown, the electronic device 180 may include at least a processor 181 and a memory 182. The memory 182 stores program codes, and when the program codes are executed by the processor 181, the processor 181 executes the steps of any of the image segmentation methods of nuclear magnetic resonance images in the above embodiments of the present application.

[0405] Refer to the following Fig.19 hereinafter describes the electronic device 190 according to this embodiment of the present application. Fig.19 The electronic device 190 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0406] like Fig.19 The electronic device 190 is in the form of a general electronic device. The components of the electronic device 190 may include but are not limited to: at least one processing unit 191, at least one storage unit 192, and a bus 193 connecting different system components (including the storage unit 192 and the processing unit 191).

[0407] Bus 193 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0408] The storage unit 192 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1921 and / or a cache memory 1922 , and may further include a read-only memory (ROM) 1923 .

[0409] The storage unit 192 may also include a program / utility 1925 having a set (at least one) of program modules 1924, such program modules 1924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0410] The electronic device 190 may also communicate with one or more external devices 194 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable objects to interact with the electronic device 190, and / or communicate with any device that enables the electronic device 190 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 195. Furthermore, the electronic device 190 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 196. As shown, the network adapter 196 communicates with other modules for the electronic device 190 via a bus 193. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 190, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0411] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0412] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0413] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0414] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0415] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for segmenting a nuclear magnetic resonance image, characterized in that: The method comprises: Acquire a nuclear magnetic resonance image of a human head, and input the nuclear magnetic resonance image into a trained segmentation model; Based on the trained segmentation model, feature extraction is performed on the nuclear magnetic resonance image to determine a feature map including image features of the nuclear magnetic resonance image; according to the resolution conversion parameter determined in the training process of the trained segmentation model, dimension conversion processing is performed on the feature map; up-sampling processing is performed on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions; The target MRI image is presented in a display interface, and different brain regions in the target MRI image are identified.

2. The method according to claim 1, characterized in that The resolution conversion parameters include resolution conversion parameters corresponding to the first resolution normalization module in the trained segmentation model and resolution conversion parameters corresponding to the second resolution normalization module in the trained segmentation model; The step of performing dimension conversion processing on the feature map according to the resolution conversion parameter determined in the training process of the trained segmentation model includes: Performing dimensionality reduction processing on the feature map according to a resolution conversion parameter corresponding to the first resolution normalization module; According to the resolution conversion parameter corresponding to the second resolution normalization module, the feature map after the dimensionality reduction processing is increased in dimension.

3. The method according to claim 1, characterized in that The segmentation model is trained in the following way: Inputting training samples in the sample set into an initial segmentation model; the training samples include sample images containing a human head and sample labels corresponding to the sample images; Based on the initial segmentation model, feature extraction is performed on the sample image to determine a sample feature map including image features of the sample image; according to a resolution conversion parameter corresponding to the initial segmentation model, dimension conversion is performed on the sample feature map; upsampling is performed on the sample feature map after the dimension conversion to obtain a nuclear magnetic resonance reference image; the nuclear magnetic resonance reference image includes a brain structure segmented from the sample image, and the brain structure includes a plurality of brain regions; A loss value is determined according to the nuclear magnetic resonance reference image and the sample label, and model parameters of the initial segmentation model are adjusted according to the determined loss value until the determined loss value is within a preset range; wherein the model parameters of the initial segmentation model include resolution conversion parameters.

4. The method according to claim 3, characterized in that The sample label corresponding to the sample image is determined according to the following method: Inputting the sample image into a trained fully convolutional neural network model; Determine the sample label corresponding to the sample image through a trained full convolutional neural network model; wherein the number of encoder and / or decoder layers in the full convolutional neural network model is greater than the number of encoder and / or decoder layers in the segmentation model.

5. The method according to claim 4, characterized in that The trained sample segmentation model is obtained through multiple iterative training processes; Among them, in each training process except the initial training process, the input samples of the fully convolutional neural network model include initial samples and target samples, the sample labels corresponding to the initial samples are pre-configured, and the sample labels corresponding to the target samples are output by the fully convolutional neural network model in the last iterative training process.

6. The method according to claim 5, characterized in that The target sample input during each iterative training process is determined as follows: Acquire a candidate sample set, wherein the candidate sample set includes at least one candidate sample that is not configured with a sample label; For any candidate sample, determine the sample label output by the full convolutional neural network model of the candidate sample during the previous multiple iterative training processes; determine the uncertainty parameter of the sample label corresponding to the candidate sample according to the sample label output by the full convolutional neural network model during the previous multiple iterative training processes; wherein the value of the uncertainty parameter is negatively correlated with the number of identical sample labels in the sample labels output by the full convolutional neural network model during the previous multiple iterative training processes; According to the uncertainty parameters of the sample labels corresponding to the candidate samples, the target samples are screened from the candidate sample set.

7. The method according to claim 3, characterized in that The determining of the loss value according to the nuclear magnetic resonance reference image and the sample label includes: Acquire multiple label types corresponding to the nuclear magnetic resonance reference image output by the initial segmentation model; For any label type, determine the probability value of each pixel in the NMR reference image being the label type; determine the probability value of the NMR reference image being the label type based on the probability value of each pixel in the NMR reference image being the label type; and determine the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type; determine the probability value of the sample label corresponding to the sample image being the label type based on the probability value of each pixel in the sample image corresponding to the NMR reference image being the label type; The loss value between the NMR reference image and the sample label is determined according to the preset weight parameters, the probability value corresponding to each label type of the NMR reference image, and the probability value corresponding to each label type of the sample label.

8. An electronic device, characterized in that: The device comprises at least one processor and at least one memory; The memory stores program codes, and when the program codes are executed by the processor, the processor performs the following process: Acquire a nuclear magnetic resonance image of a human head, and input the nuclear magnetic resonance image into a trained segmentation model; Based on the trained segmentation model, feature extraction is performed on the nuclear magnetic resonance image to determine a feature map including image features of the nuclear magnetic resonance image; according to the resolution conversion parameter determined in the training process of the trained segmentation model, dimension conversion processing is performed on the feature map; up-sampling processing is performed on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions; The target MRI image is presented in a display interface, and different brain regions in the target MRI image are identified.

9. The electronic device according to claim 8, characterized in that: The resolution conversion parameters include resolution conversion parameters corresponding to the first resolution normalization module in the trained segmentation model and resolution conversion parameters corresponding to the second resolution normalization module in the trained segmentation model; the electronic device is specifically used for: Performing dimensionality reduction processing on the feature map according to a resolution conversion parameter corresponding to the first resolution normalization module; According to the resolution conversion parameter corresponding to the second resolution normalization module, the feature map after the dimensionality reduction processing is increased in dimension.

10. An image segmentation device for a nuclear magnetic resonance image, characterized in that: The device comprises: An acquisition module, used for acquiring a nuclear magnetic resonance image of a human head, and inputting the nuclear magnetic resonance image into a trained segmentation model; a segmentation module, which extracts features from the nuclear magnetic resonance image based on the trained segmentation model to determine a feature map including image features of the nuclear magnetic resonance image; performs dimension conversion processing on the feature map according to a resolution conversion parameter determined in the training process of the trained segmentation model; performs upsampling processing on the feature map after the dimension conversion processing to obtain a target nuclear magnetic resonance image; the target nuclear magnetic resonance image includes a brain structure segmented from the nuclear magnetic resonance image, and the brain structure includes a plurality of brain regions; The presentation module is used to present the target MRI image in a display interface and identify different brain regions in the target MRI image.

Images