A method and system for brain magnetic resonance segmentation based on combined dual-task autoencoder

Abstract

The application discloses a brain magnetic resonance segmentation method and system based on a combined double-task self-encoder, and registration is performed on a segmentation training set of a downstream segmentation task, then the segmented training set after registration is centrally cropped, then the data after central cropping is resampled to obtain feature data; a pre-trained self-encoder is used to perform feature extraction on the obtained feature data to obtain basic features, and a decoding operation is performed on the obtained basic features to obtain a segmentation result; the network segmentation model is trained by using the decoded segmentation result and the corresponding segmentation training set, the trained network segmentation model is used for segmentation of MR images, the double model is used for segmentation, the precision of the segmentation result can be greatly improved, the combined double-task framework of the pixel level and the object level is adopted, the model learns the pixel level details and the object level distinguishing information respectively, the modal information is fused in a shared parameter mode, and the segmentation result of the model is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a brain magnetic resonance segmentation method and system based on a combined double-task self-encoder. BACKGROUND

[0002] Medical image segmentation plays an important role in computer-aided diagnosis and treatment, and can help doctors analyze disease data. The task is to accurately delineate the target region such as organs, lesions, tissues, etc. at the pixel level in medical imaging. At present, the main method for medical image segmentation in clinical practice is still manual annotation, which requires strong medical expert knowledge and experience. Generally, doctors perform annotation, and ordinary people are difficult to perform this task, which leads to low annotation data output efficiency. At the same time, due to the noise of machine imaging, the divergence of subjective judgment of experts, and the fatigue caused by a large amount of repetitive labor, manual annotation is prone to subjective human error. Therefore, there is a need to develop accurate automatic medical image segmentation algorithms. These algorithms have strong objectivity compared with human annotation, and can quickly perform batch image annotation, greatly reducing the labor of doctors.

[0003] At present, there is a surge of interest in the study of self-supervised methods, but there are still few works in the medical field using self-supervised methods. Some studies point out that self-supervised learning can be directly applied to the medical field, because unlabeled medical images contain valuable information about organ structure, and self-supervision enables the model to derive concepts about these structures without additional annotation costs. Unlike natural images, medical images have 3D properties, that is, they are presented in sequence. At present, many studies on self-supervised methods convert 3D imaging tasks into 2D by extracting slices along any axis (such as axial dimension). However, relying on 2D context to obtain data representation from 3D images is a suboptimal solution, which will reduce the performance of downstream tasks.

[0004] Deep learning models are usually trained using the supervised learning paradigm, in which the model learns to map inputs (such as magnetic resonance images or health records) to outputs. In order for the model to learn the relevant patterns in the data, training through supervised learning requires large datasets, each of which has corresponding label information. Deep learning has achieved great success on supervised models. However, compared with the heavy work of building annotated datasets, more emphasis is placed on building and testing models. This is partly because, for most medical tasks, expert annotation of large-scale patient multi-modal data is non-trivial, expensive and time-consuming, and is associated with privacy disclosure risks, even semi-automatic software tools may not be able to sufficiently reduce annotation costs.

[0005] On the other hand, unlike natural images, although there are individual differences, because the human body is depicted, the physical structure of medical images is relatively stable, and the image presents natural and consistent context information, and the lesion also has its specific texture and appearance. The self-supervised agent task is used to learn the basic pattern of human anatomy, and in this case, the accurate segmentation result cannot be obtained in the self-supervised learning task, thereby affecting the discriminative accuracy of the encouraged model. SUMMARY

[0006] The purpose of the present application is to provide a brain magnetic resonance segmentation method and system based on a combined double-task autoencoder to overcome the problem that the prior art cannot obtain accurate segmentation results and affects the discriminative accuracy of the encouraged model.

[0007] A brain magnetic resonance segmentation method based on a combined double-task autoencoder, comprising the following steps:

[0008] The segmentation training set of the downstream segmentation task is registered, and then the registered segmentation training set is center cropped, and then the center cropped data is resampled to obtain feature data;

[0009] The pre-trained autoencoder is used to extract features from the obtained feature data to obtain basic features, and the obtained basic features are decoded to obtain a segmentation result;

[0010] The decoded segmentation result and the corresponding segmentation training set are used to train the network segmentation model, and the trained network segmentation model is used for MR image segmentation.

[0011] Preferably, the training process of the pre-trained autoencoder is as follows: collect pre-training images as a pre-training set, convert the pre-training set data into a brain imaging data structure, and standardize the data structure of the pre-training set converted into the brain imaging data structure to the same template, then center crop the pre-training set data after registration to obtain pre-training feature data, and use the pre-training feature data to train the autoencoder.

[0012] Preferably, the data structure of the pre-training set is standardized to the same template using the Clinica platform.

[0013] Preferably, two positive correlation views of each sample in the same batch after data enhancement are obtained by randomly rotating the obtained same batch of pre-training feature data, and a random mask operation is performed on each positive correlation view to obtain a positive correlation feature map with a shielded partial patch; the positive correlation feature map after data enhancement and random mask operation is input into the autoencoder network for feature extraction, and the reconstructed image patch and contrast encoding feature are obtained through the pixel-level prediction head and the object-level prediction head, respectively, and the pre-training of the autoencoder is performed using the self-supervised information of the pre-training image itself.

[0014] Preferably, in the pre-training process of the autoencoder, the network parameters are optimized using a back propagation strategy, and a loss function is used to help training. According to the value of the loss function, the network parameters are updated so that the loss function continuously decreases until it converges to a set value, at which point the training is completed and the pre-training of the autoencoder is completed.

[0015] Preferably, the data structure of the pre-training set is standardized to the same template using the Clinica platform, which includes the following steps:

[0016] A maximum adjacent rectangle frame is calculated for the maximum foreground region of all modalities after mutual matching and registration, and the region with a value of "0" is excluded. The spatial size of each sample is unified to a consistent level after registration of the entire pre-training set, and then resampling is performed to obtain feature data, wherein the target space of resampling is obtained by averaging the entire data set.

[0017] Preferably, for single-modality data, the autoencoder is directly loaded as a feature extraction encoder, and the low-level semantics in the encoding stage are connected to the high-level semantics in the decoding stage at the same down-sampling rate through a U-shaped network structure to obtain a segmentation result.

[0018] Preferably, for multi-modality data, a simple modality-shared encoder is used to input different modality data into the encoder with shared parameters to capture common features of each modality and obtain a segmentation result.

[0019] Preferably, the encoding output of multi-modality is decoded, and the low-level semantics in the encoding stage are connected to the high-level semantics in the decoding stage at the same down-sampling rate through a cross-layer connection to finally obtain a segmentation result.

[0020] A brain magnetic resonance segmentation method based on a combined double-task autoencoder includes a data preprocessing module, a self-supervised module, and a segmentation module:

[0021] The data preprocessing module is used to register the segmentation training set of the downstream segmentation task, then the center crop is performed on the registered segmentation training set, and then the feature data is obtained by resampling the center-cropped data;

[0022] The self-supervised module is used for feature extraction on the obtained feature data to obtain basic features, and decoding operation is performed on the obtained basic features to obtain a segmentation result.

[0023] The segmentation module trains a network segmentation model by using the decoded segmentation result and the corresponding segmentation training set, and performs segmentation on the MR image by using the trained network segmentation model.

[0024] Compared with the prior art, the present application has the following beneficial technical effects:

[0025] The present application provides a brain magnetic resonance segmentation method based on a combined double-task autoencoder, which is registered to the segmentation training set of the downstream segmentation task, then the registered segmentation training set is center cropped, and then the center cropped data is resampled to obtain feature data; the obtained feature data is extracted by using a pre-trained autoencoder to obtain basic features, and decoding operation is performed on the obtained basic features to obtain a segmentation result; a network segmentation model is trained by using the decoded segmentation result and the corresponding segmentation training set, and the trained network segmentation model is used for segmentation of the MR image, and the double model is used for segmentation, which can greatly improve the accuracy of the segmentation result.

[0026] Further, a combined double-task framework of pixel level and object level is adopted, so that the model learns pixel level details and object level distinguishing information respectively, for multi-modal data in the segmentation task, a modal based autoencoder loading strategy is proposed, modal information is fused by sharing parameters, and the model segmentation result is improved.

[0027] Further, for multi-modal data, a dedicated normalization layer is used in each branch to distinguish the data distribution of each branch, the modal private information is retained, the data processing amount is reduced, and the image processing accuracy is improved.

[0028] Further, by jointly using cross-entropy loss, SoftDice loss and deep supervision loss, the gradient back propagation is promoted, the model convergence is strengthened, and the model training effect is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is the implementation flowchart of the brain magnetic resonance segmentation method in the embodiment of the present application.

[0030] Figure 2 is the network segmentation model structure diagram based on the combined double-task autoencoder in the embodiment of the present application.

[0031] Figure 3 is the focused masking autoencoder diagram in the combined double-task model in the self-supervised stage in the embodiment of the present application.

[0032] Figure 4It is a self-supervised phase combined double task model based on a contrast-based self-encoder structure schematic diagram in the embodiment of the application.

[0033] Figure 5 It is a modal-based downstream brain magnetic resonance segmentation task network framework diagram in the embodiment of the application.

[0034] Figure 6 It is a feature extraction layer diagram of a modal-based downstream brain magnetic resonance segmentation encoder in the embodiment of the application.

[0035] Figure 7 It is a segmentation effect diagram of a brain magnetic resonance segmentation method model in the embodiment of the application. DETAILED DESCRIPTION

[0036] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] As shown in Figure 1 , Figure 2 The present application provides a brain magnetic resonance segmentation method based on a combined double task self-encoder, which specifically comprises the following steps:

[0039] S1, data preprocessing: collecting pre-training images as a pre-training set, converting the pre-training set data into a brain imaging data structure, and registering the data structure of the pre-training set converted into the brain imaging data structure to the same template, and then performing center cropping operation on the registered pre-training set data to obtain pre-training feature data.

[0040] In the present application, the data structure of the pre-training set is standardized to the same template by using the Clinica platform.

[0041] The self-encoder pre-training process: two positive correlation views of each sample in the same batch after data enhancement are obtained by random rotation operation on the obtained same batch of pre-training feature data, and random mask operation is performed on each positive correlation view to obtain a positive correlation feature map with a shielded part patch; the positive correlation feature map after data enhancement and random mask operation is input into the self-encoder network for feature extraction operation, and the reconstructed image patch and the contrast encoding feature are obtained through the pixel-level prediction head and the object-level prediction head respectively, and the self-encoder is pre-trained using the self-supervised information of the pre-training image itself.

[0042] In the pre-training process of the self-encoder, the network parameters are optimized using the back propagation strategy, and the loss function is used to help training. According to the value of the loss function, the network parameters are updated so that the loss function continuously decreases until it converges to a set value. At this time, the training is completed and the pre-training of the self-encoder is completed.

[0043] Specifically, in the pre-training phase of the self-encoder, the public data set is used as the pre-training data set, and the pre-training data set is divided into a pre-training set.

[0044] The pre-training set data is converted into brain imaging data structure, and the Clinica platform is used to standardize the pre-training set data to the same template, which specifically includes the following steps:

[0045] A maximum adjacent rectangle frame is calculated for the maximum foreground area of all modalities matched and registered, and the area of "0" is excluded, and the spatial size of each sample is unified to a consistent level after registration of the entire pre-training set, and then resampling is performed to obtain feature data, wherein the target space of resampling is obtained by averaging the entire data set;

[0046] Specifically, the pre-training set is used for training, and the label information of the data set is not used in the training process. Among them, in order to make the training more stable, an oversampling strategy is adopted to ensure that at least one third of the data in a batch contains foreground.

[0047] The specific process of self-encoder training is: the self-encoder is pre-trained by using the combination of pixel-level and object-level dual-agent tasks. The specific process is: for a given 3D voxel volume input sample X R H ×W×D×C Random rotation operation data enhancement: the input data instance is randomly transformed into two positive correlation views of the same sample through data enhancement, and finally the data after enhancement contains 2N data points; the rotation enhancement formula is:

[0048]

[0049] r=RandomChoices(R),R={0°,90°,180°,270°}

[0050] In the formula: r is the random rotation angle, and k represents the sample number of the batch size N.

[0051] For the enhanced example and First, it is reshaped into a series of flat 3D patches. To preserve positional information, and considering the brain's bilateral symmetry, symmetrical positional encoding is incorporated into the patch embedding. The formula for calculating the positional encoding is as follows:

[0052] pos=h·x-|w / 2-y|+w / 2+d 2 ·z

[0053]

[0054]

[0055] In the formula: dim is the dimension of the patch encoding, pos is the position of the patch embedding at coordinates (x, y, z), and i represents different dimensions of the positional encoding. For patch embeddings at the same position, an alternating encoding method of sine and cosine coding is used. Since the values ​​of sine and cosine are between -1 and 1, adding positional encoding to the patch embedding does not cause significant distortion. The final input of the autoencoder is the sum of the positional encoding and the patch embedding.

[0056] The patch data is randomly masked, and the visible patch regions are fed into the encoder. The 3D patch sequence is then projected into a fixed-dimensional space through an embedding layer. To more efficiently model the interaction of patch embeddings, a window of size M×M×M is used to uniformly divide the input features of size H′×W′×D′ into non-overlapping windows, and local self-attention is calculated within each region. The windows are then offset by (M / 2, M / 2, M / 2) voxels, so that the features from the different previously divided windows appear in the same window after the shift. Calculating local self-attention within the window region at this point enables information exchange between the different windows.

[0057] like Figure 3 , Figure 4 As shown, the output of the autoencoder after feature extraction is decoded, and finally the reconstructed image patch and contrast-coded features are obtained through pixel-level and object-level prediction heads.

[0058] The training data is used for training, and during the training process, the parameters of the network are optimized using a back propagation strategy, and a loss function is used to help training, wherein the loss function used includes a focus reconstruction loss and a contrast loss; the focus reconstruction loss and the contrast loss are used to help training, the parameters of the network are optimized through back propagation, and the model is encouraged to learn basic features of an image.

[0059] The focus reconstruction loss is based on the relatively stable characteristics of brain tissue structure, and the gradient of each voxel is calculated to obtain a gradual change image in each direction, and the calculation formula is as follows:

[0060] G i =I*D i ,i∈{x,y,z}

[0061] In the formula, I is an input feature, D i is a filter in the i direction, and * represents convolution operation.

[0062] Therefore, the gradient direction θ of each voxel is obtained, and the specific formula is as follows:

[0063]

[0064] The gradient amplitude is:

[0065]

[0066] For each image patch, a 2D directional gradient histogram is created, each voxel is traversed, and the gradient direction of the voxel is determined to fall in which interval of the X and Y axes, and the gradient amplitude of the voxel is accumulated on the Y axis of the interval. After the traversal is completed, the vector representing the size of the histogram needs to be normalized. The importance of all the masked image patches in the entire brain tissue is obtained, and the calculation formula is as follows:

[0067]

[0068] In the formula: represents the average value of the histogram, and N is the number of randomly masked image patches.

[0069] According to the importance of the directional gradient histogram, different weights are applied when measuring the pixel difference between the restored image region and the original image, so as to encourage the model to pay more attention to important regions. The calculation formula of the focus reconstruction loss function is as follows:

[0070]

[0071] The contrast reconstruction loss is to set a pair of enhanced samples in the same batch as positive examples z i and z jThe other 2(N-1) enhanced samples in the same batch are taken as negative samples. The mutual information between the two vectors is calculated by the cosine similarity, and the cosine similarity formula is:

[0072]

[0073] Therefore, the contrast loss calculation formula is:

[0074]

[0075] According to the value of the loss function, the network parameters are updated so that the loss function continuously decreases until it converges to a small value. At this time, the training is completed, and the trained pre-trained autoencoder is saved.

[0076] Collect the training images of the downstream segmentation task as the segmentation training set, register the segmentation training set, then perform intermediate cropping on the registered segmentation training set, and then perform resampling to obtain feature data; input the obtained feature data into the self-supervised stage to obtain basic features, and then perform decoding operation on the extracted basic features, and simultaneously connect the low-level semantics processed by convolution with the high-level semantics in the decoding stage under the same down-sampling ratio through cross-layer connection, and finally obtain the segmentation result by decoding; use the decoded segmentation result and the corresponding segmentation training set to train the network segmentation model, and use the trained network segmentation model to segment the MR image.

[0077] In the downstream brain MRI segmentation stage, for single modality data, the autoencoder is directly loaded as a feature extraction encoder, and the low-level semantics in the encoding stage are connected with the high-level semantics in the decoding stage under the same down-sampling ratio through cross-layer connection by using a U-shaped network structure to obtain the segmentation result. For multi-modal data, a simple modal shared encoder is used, and different modal data is input into the encoder with shared parameters to capture common features of each modality, and a segmentation result is obtained.

[0078] For modal private information, after convolution, separate normalization operation is performed on the multi-modal, and the modal private feature is separately counted, and the specific formula is:

[0079]

[0080] In the formula: u L , represents the mean and variance of the whole sample. ò is a very small constant to prevent the denominator from being 0. α m ,β m are trainable parameters, which are scaling factors and translation parameters in affine transformation, respectively, used to restore the expression ability of data. Through modal privatization (α m ,β m), to achieve the role of distinguishing statistical modal information.

[0081] The multi-modal encoded output is decoded, and the low-level semantics in the encoding stage are connected with the high-level semantics in the decoding stage at the same down-sampling rate through cross-layer connection, and finally the segmentation result is obtained.

[0082] In the network segmentation model training process, the parameters of the network are optimized using the back propagation strategy, and the loss function is used to help training, wherein the loss function used includes cross entropy loss, SoftDice loss, and full resolution deep supervision loss; cross entropy loss, SoftDice loss, and full resolution deep supervision loss are used to help training, and the parameters of the network are optimized by back propagation.

[0083] Cross entropy is the most commonly used loss in image segmentation algorithms, which compares each pixel with the true value map one by one, and its formula is as follows:

[0084]

[0085] In the formula: D x W x H is the number of pixels of the entire three-dimensional image, y i ∈{0,1} is the true label of the i-th element, wherein 0 is the background and 1 is the foreground, p i ∈[0,1] represents the probability that the network predicts that the pixel belongs to the foreground.

[0086] The formula of SoftDice loss is as follows:

[0087]

[0088] In the formula: ε is a very small constant to prevent the denominator from being 0.

[0089] The decoding layer of each stage of the network is taken as the intermediate output, and the output is up-sampled according to the down-sampling rate of the stage, and the Side loss is introduced in the form of full resolution for deep supervision. The final loss function formula is as follows:

[0090]

[0091] In the formula: P is the predicted probability map, Y is the true value map, g(*, u) represents up-sampling with u as the rate, λ i is a hyperparameter for balancing the losses of different intermediate layers, and N is the number of intermediate layers.

[0092] According to the value of the loss function, the network parameters are updated so that the loss function continuously decreases until it converges to a small value, at which time the training is completed and the trained network model is saved; the saved trained model is used to form a brain magnetic resonance segmentation model.

[0093] A brain magnetic resonance segmentation system based on a combined double-task autoencoder includes a data preprocessing module, a self-supervised module and a segmentation module:

[0094] The data preprocessing module is used for registering the segmentation training set of the downstream segmentation task, then performing intermediate cropping on the registered segmentation training set, and then resampling the center cropped data to obtain feature data;

[0095] The self-supervised module is used for feature extraction on the obtained feature data to obtain basic features, and decoding operation on the obtained basic features to obtain a segmentation result;

[0096] The segmentation module uses the decoded segmentation result and the corresponding segmentation training set to train a network segmentation model, and uses the trained network segmentation model to segment the MR image.

[0097] The brain magnetic resonance segmentation method based on the combined double-task autoencoder of the application designs a reconstruction proxy task for brain tissue prior knowledge to focus on important image features; in order to learn the basic mode of the brain region anatomical structure, a combined double-task framework suitable for pixel-level and object-level of brain magnetic resonance imaging is proposed, so that the model learns pixel-level details and object-level distinguishing information respectively; by jointly using the focus reconstruction loss and the contrast loss, the gradient back propagation is promoted, the model convergence is strengthened, and the model learning basic features is further encouraged.

[0098] In the downstream task, according to the number of modalities, the modal-based autoencoder loading strategy (MALS) enables the autoencoder to be directly loaded as a feature extraction encoder in the segmentation for single modality data, and a U-shaped network is used to obtain the segmentation result. For multi-modal data, an encoder with shared feature extraction parameters is used to extract modality public information, but for modality private information, separate normalization is performed on the multi-modal data, and the modality private features are counted separately.

[0099] By jointly using the cross-entropy loss, the SoftDice loss and the deep supervision loss, the gradient back propagation is promoted, the model convergence is strengthened, and the model training effect is further improved;

[0100] The application obtains competitive Dice and HD results in the downstream segmentation task on three public data sets, which is better than the current popular self-supervised medical image segmentation model.

[0101] Embodiment

[0102] A brain magnetic resonance segmentation method based on a combined double-task autoencoder includes the following steps:

[0103] Self-supervised learning of the autoencoder: preprocessing the 3D medical source data to make it suitable for model training. The specific workflow is as follows:

[0104] (1.1), using two sets of public data sets as pre-training data sets;

[0105] (1.2), converting the data set of step (1.1) into a brain imaging data structure, using the Clinica platform to standardize the data registration to the same template;

[0106] (1.3), calculating a maximum adjacent rectangular frame for the maximum foreground area of all modalities, excluding the area of "0" in the foreground area, and uniformly resizing the space of each sample to a consistent level for the entire training set after registration, so that the convolution kernel traverses the data to extract features with the same receptive field, thereby resampling, wherein the target space of resampling is obtained by averaging the entire data set;

[0107] (1.4), dividing the entire data set processed in step (1.3) into a training set, and the training process does not use any label information of the data set.

[0108] The reconstruction proxy task is designed according to the prior knowledge of brain tissue to focus on important image features, and a proxy task based on contrast coding is added on this basis to form a combined double-task framework of pixel level and object level. The specific workflow is as follows:

[0109] (2.1), for the data set obtained in step (1.4), for the input sample X∈R H×W×D×C , data augmentation by random rotation is performed to obtain two positive views of each enhanced example, that is

[0110] (2.2), for the enhanced example of step (2.1), reshape it into a series of flat 3D patches, in order to ensure the position information, and according to the structure of the brain having the characteristic of left-right symmetry, add symmetric position coding in patch embedding, that is pos=h·x-|w / 2-y|+w / 2+d 2 ·z;

[0111] (2.3), for the feature patch sequence with position coding obtained in step (2.2), perform random mask operation, and send the visible patch area to the encoder;

[0112] (2.4), use paired window self-attention calculation modules and moving window self-attention calculation modules in the encoder to more efficiently model the interaction of patch embedding.

[0113] (2.5), decode the encoder output obtained in step (2.4), and finally obtain the reconstructed image patch and contrast coding feature through the prediction head of pixel level and object level.

[0114] The acquired autoencoder training process adopts focus reconstruction loss and contrast loss to promote gradient backpropagation, strengthen model convergence, and further encourage the model to learn basic features;

[0115] The 3D medical source data is pre-processed to adapt to the training of the model. The specific workflow is as follows:

[0116] (4.1), three groups of public data sets are used as training data sets;

[0117] (4.2), the data set described in step (4.1) is converted into a brain imaging data structure, and the Clinica platform is used to standardize the data registration to the same template;

[0118] (4.3), a maximum adjacent rectangular frame is calculated for the maximum foreground area of all modalities, the area of the foreground area is excluded, and the space size of each sample is unified to the same level after registration of the entire training set, so that the convolution kernel traverses the data to extract features with the same receptive field, thereby resampling, wherein the target space of resampling is obtained by averaging the entire data set;

[0119] (4.4), the data set processed in step (4.3) is divided into a training set and a test set.

[0120] For different modal data, a modal autoencoder loading strategy (MALS) is proposed, as shown in Figure 5 、 Figure 6 The specific workflow is as follows:

[0121] (5.1), for single modal data, directly load the autoencoder as a feature extraction encoder.

[0122] (5.2), for multi-modal data, a simple modal shared encoder is used, which inputs different modal data into an encoder with shared parameters to capture common features of each modality.

[0123] (5.3), for the modal private information of step (5.2), after convolution, separate normalization operation is performed on the multi-modal data to separately calculate the modal private features.

[0124] The basic features extracted by the autoencoder are decoded, and the low-level semantics in the encoding stage are connected with the high-level semantics in the decoding stage with the same down-sampling rate through cross-layer connection, and finally the segmentation result is obtained.

[0125] For the network segmentation model, cross-entropy loss, SoftDice loss and full-resolution deep supervision loss are used in the training process to promote gradient backpropagation, strengthen model convergence and further improve training effect.

[0126] For the trained network segmentation model, a test image is taken as input to obtain an automatic segmentation result, as shown in Figure 7 The specific workflow is as follows:

[0127] The application converts the training set data into brain imaging data structure in the self-supervised stage, normalizes the data registration to the same template, then performs center cropping operation on the registered training set, and finally performs resampling to obtain feature data; the same batch of feature data is obtained, and the two positive correlation views of each sample in the same batch after data enhancement are obtained by random rotation operation, and random mask operation is performed on each view to obtain the feature of the shielded part patch; the positive correlation feature map after data enhancement and random mask operation is input into the autoencoder network for feature extraction operation, and the reconstructed image patch and contrast coding feature are obtained through the pixel-level prediction head and the object-level prediction head respectively, finally the self-encoder is trained using the self-supervision information of the image itself, the symmetric position coding is introduced according to the prior knowledge of brain magnetic resonance imaging, so that the symmetric position has the same position information, and considering the smoothness of medical images relative to natural images, the importance of the features of different regions is classified according to the distribution of local intensity gradient of three-dimensional voxel direction gradient histogram, and different weights are applied when the reconstruction loss is measured by measuring the pixel difference between the recovered image region and the original image, so as to encourage the model to pay more attention to important areas, in order to learn the basic pattern of brain region anatomy, a combined double-task framework of pixel-level and object-level customized for brain magnetic resonance imaging is proposed, so that the model learns pixel-level details and object-level distinguishing information respectively, for multi-modal data in the downstream brain MRI segmentation task, a modal-based autoencoder loading strategy is proposed, which fuses modal information through the way of sharing parameters, and improves the model segmentation result.

Claims

1. A method for brain magnetic resonance segmentation based on combined dual-task autoencoder, characterized in that, The method comprises the following steps: The segmentation training set of the downstream segmentation task is registered, then the registered segmentation training set is center cropped, and then the center cropped data is resampled to obtain feature data; The pre-trained autoencoder is used to extract features from the obtained feature data to obtain basic features, and the obtained basic features are decoded to obtain a segmentation result; The decoded segmentation result and the corresponding segmentation training set are used to train a network segmentation model, and the trained network segmentation model is used for segmentation of MR images; The specific process of training the pre-trained autoencoder is as follows: pre-training images are collected as a pre-training set, the pre-training set data is converted into a brain imaging data structure, and the data structure of the pre-training set converted into the brain imaging data structure is registered and standardized to the same template, then the pre-training set data after registration is center cropped to obtain pre-training feature data, and the pre-training feature data is used to train the autoencoder; The same batch of pre-training feature data is randomly rotated to obtain two positive correlation views of each sample in the same batch after data enhancement, and a random mask operation is performed on each positive correlation view to obtain a positive correlation feature map with a shielded patch; The positive correlation feature map after data enhancement and random mask operation is input into the autoencoder network for feature extraction, and the reconstructed image patch and the contrast encoding feature are obtained through the pixel-level prediction head and the object-level prediction head respectively, and the pre-training of the autoencoder is performed using the self-supervision information of the pre-training images.

2. The method of claim 1, wherein, The data structure of the pre-training set is registered and standardized to the same template by using the Clinica platform.

3. The method of claim 1, wherein the method is based on a combined dual-task autoencoder. In the pre-training process of the autoencoder, the network parameters are optimized using the back propagation strategy, and the loss function is used to help training. According to the value of the loss function, the network parameters are updated so that the loss function continuously decreases until it converges to a set value. At this time, the training is completed, and the pre-training of the autoencoder is completed.

4. The method of claim 2, wherein the method is based on a combined dual-task autoencoder. The data structure of the pre-training set is registered and standardized to the same template by using the Clinica platform, which comprises the following steps: A maximum adjacent rectangle frame is calculated for the maximum foreground region of all modalities after mutual matching and registration, and the region of "0" is excluded. The spatial size of each sample of the registered pre-training set is uniformly adjusted to a consistent level, and then resampling is performed to obtain feature data, wherein the target space of resampling is obtained by averaging the entire data set.

5. The method of brain magnetic resonance image segmentation based on combined dual-task autoencoder according to claim 1, characterized in that, For single modality data, the autoencoder is directly loaded as a feature extraction encoder, and the low-level semantics in the encoding stage are connected to the high-level semantics in the decoding stage at the same down-sampling rate through a U-shaped network structure to obtain a segmentation result.

6. The method of brain magnetic resonance image segmentation based on combined dual-task autoencoder according to claim 1, characterized in that, For multi-modal data, a simple modal shared encoder is used to input different modal data into the encoder with shared parameters to capture common features of each modality and obtain a segmentation result.

7. The method of brain magnetic resonance image segmentation based on combined dual-task autoencoder according to claim 6, characterized in that, The encoding output of the multi-modal data is decoded, and the low-level semantics in the encoding stage are connected to the high-level semantics in the decoding stage at the same down-sampling rate through a cross-layer connection to finally obtain a segmentation result.

8. A brain magnetic resonance segmentation system based on the combined dual-task autoencoder of claim 1, wherein, The method comprises a data preprocessing module, a self-supervision module and a segmentation module: The data preprocessing module is configured to perform registration on a segmentation training set of a downstream segmentation task, then perform center cropping on the registered segmentation training set, and then perform resampling on the center cropped data to obtain feature data. The self-supervised module is configured to perform feature extraction on the obtained feature data to obtain basic features, and perform decoding on the obtained basic features to obtain a segmentation result. The segmentation module is configured to train a network segmentation model by using the decoded segmentation result and the corresponding segmentation training set, and perform segmentation on an MR image by using the trained network segmentation model.

Images