Brain multiple sclerosis diseased region change detection method

Abstract

The invention provides a brain multiple sclerosis diseased region change detection method. The method comprises the following steps that 1, brain nuclear magnetic resonance images of a brain multiple sclerosis patient at multiple times are obtained and preprocessed; 2, constructing a model for detecting the change of a brain multiple sclerosis lesion part, and training the model; step 3, detecting changes of the pathological regions of the brain multiple sclerosis, obtaining change characteristics of each pathological region of the brain multiple sclerosis patient at each time based on a plurality of time ranges, and obtaining a pathological position detection map of the brain multiple sclerosis patient at each time; and step 4, generating and outputting a change trend chart of each lesion part of the brain multiple sclerosis patient in a plurality of time ranges. According to the invention, the change of the brain multiple sclerosis in a long time can be effectively tracked, and a doctor can be better assisted to carry out tracking analysis and diagnosis and treatment on the condition.

Description

Technical Field

[0001] The invention relates to a method for detecting changes in brain multiple sclerosis lesion sites, and belongs to the technical field of image processing. Background Art

[0002] The symptoms of multiple sclerosis in the brain are highly hidden, the diagnosis is complex, and it has the characteristic of "multiple recurrences over time", that is, it has a long course of disease, with obvious relapses and remissions, and requires long-term observation, diagnosis and treatment. Therefore, the use of image processing and artificial intelligence technology to assist in the diagnosis and long-term tracking of the disease has become one of the cutting-edge technologies in recent years.

[0003] The use of image processing and artificial intelligence technology to detect the location of multiple sclerosis lesions in the brain is currently in its early stages, and the existing technical means are mainly divided into two types: single-time detection and dual-time detection. Single-time detection only uses image processing and artificial intelligence to process the patient's MRI images at a certain moment to examine the patient's lesion site at the current moment; dual-time detection compares the patient's lesion site at two times to analyze the patient's short-term course of disease. However, due to the "temporal multi-occurrence" characteristics of multiple sclerosis in the brain, the disease undergoes complex changes over a long period of time, so a detection method is needed that can detect the patient's brain lesions over a long period of time to assist in the diagnosis and clinical treatment of the disease. Summary of the invention

[0004] The present invention provides a method for detecting changes in the lesion site of multiple sclerosis in the brain, which can effectively detect the disease evolution of the lesion site of multiple sclerosis in the brain within multiple time ranges, so as to assist the diagnosis and clinical treatment of multiple sclerosis in the brain.

[0005] In order to solve the above technical problems, the present invention provides a method for detecting changes in the lesion site of multiple sclerosis in the brain, comprising the following steps: Step 1, obtaining multiple time periods of a patient with multiple sclerosis in the brain t MRI of the brain L t , for each brain MRI image L tPreprocessing is performed; step 2, a model for detecting changes in the lesion site of multiple sclerosis in the brain is constructed and trained, wherein the model for detecting changes in the lesion site of multiple sclerosis in the brain comprises a visual feature encoder, an image registration model, a time domain Transformer encoder, a multi-layer perceptron and a Transformer decoder, wherein the output end of the visual feature encoder is connected to the image registration model, the output end of the image registration model is connected to the time domain Transformer encoder and the multi-layer perceptron, and the output end of the time domain Transformer encoder is connected to the multi-layer perceptron and the Transformer decoder; step 3, each brain magnetic resonance image after preprocessing is L t All of them are input into the model for detecting changes in the lesion site of multiple sclerosis in the brain to detect changes in the lesion site of multiple sclerosis in the brain, and the change characteristics of each lesion site of the multiple sclerosis patient in the brain based on multiple time ranges at each time are obtained, and the lesion location detection map of the multiple sclerosis patient in the brain at each time is obtained. M t ; Step 4, based on the change characteristics of each lesion site in the brain of the multiple sclerosis patient at each time within multiple time ranges obtained in step 3, generate and output a change trend graph of each lesion site in the brain of the multiple sclerosis patient within multiple time ranges.

[0006] In a specific embodiment, the step 3 is specifically as follows: 3.1 Multiple time visual feature encoding: each brain magnetic resonance imaging image after preprocessing L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t ; 3.2 Alignment of visual features at multiple times: Align each visual feature map F t Input to the image registration model to obtain the aligned visual feature map at each time F t ';3.3 Temporal feature extraction of multiple times: Align each visual feature map at each time F t 'All are input into the time domain Transformer encoder to obtain the time domain feature map at each time F t ^; 3.4 Detection of multiple sclerosis lesions in the brain: Visual feature maps at each time F t 'And the time domain feature map corresponding to the time F t^All are cascaded and input into the multi-layer perceptron to obtain the lesion location detection map of the brain of multiple sclerosis patients at each time M t ; 3.5 Change detection of brain multiple sclerosis lesion sites at multiple times: Visual feature maps at each time F t 'And the time domain feature map corresponding to the time F t ^are respectively input into the Transformer decoder to obtain the change characteristics of each lesion site in the brain of patients with multiple sclerosis based on multiple time ranges at each time.

[0007] In a specific embodiment, the 3.1 preprocessed brain magnetic resonance imaging L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t , specifically as follows: the visual feature encoder includes an imaging omics model, a deep convolutional network model, and a visual Transformer encoder, and the output ends of the imaging omics model and the deep convolutional network model are connected to the visual Transformer encoder; 3.1.1 Each brain MRI image after preprocessing L t are input into the imaging omics model and the deep convolutional network model respectively; 3.1.2 The imaging omics model extracts each MRI image after preprocessing L t Radiomics features to obtain each MRI image L t The underlying information of each MRI image after preprocessing is encoded by the deep convolutional network model described in 3.1.3 L t Convolutional features to extract each MRI image L t 3.1.4 Each MRI image L t The radiomics features and convolutional features of each MRI image are cascaded and fused. L t The features of radiomics and convolution features are integrated; 3.1.5 Each MRI image L t The features that integrate radiomics features and convolutional features are input into the visual Transformer encoder for encoding to obtain a visual feature map that integrates radiomics features, convolutional features and relational features at each time.F t .

[0008] In a specific implementation, the visual Transformer encoder includes two encoding layers, the temporal Transformer encoder includes two encoding layers, and the Transformer decoder includes two decoding layers.

[0009] In a specific embodiment, each brain magnetic resonance imaging L t The preprocessing process includes the following steps: 1.1 For each brain MRI image L t Perform normalization processing; 1.2 Each brain MRI image after normalization processing L t Cropped into cells of size 60x60x60 with a stride of 20x20x20.

[0010] In a specific embodiment, the step 3.2 is as follows: first, each brain magnetic resonance imaging image after preprocessing is L t are input into the image registration model, and the co-registration method is used in the image registration model to register each preprocessed brain MRI image L t Align, and then map the visual features at each time F t are input into the image registration model, based on each brain MRI image after preprocessing L t The alignment result of each time visual feature map is used in the image registration model. F t Align to obtain the aligned visual feature map at each time F t '.

[0011] In a specific embodiment, the training of the model for detecting changes in the lesion site of multiple sclerosis in the brain is specifically as follows: obtaining a training data set, wherein the training data set includes brain magnetic resonance imaging (MRI) images of multiple sclerosis patients at multiple times and manually annotated lesion sites at each time; preprocessing each brain MRI image in the training data set, specifically as follows: normalizing each brain MRI image in the training data set; cropping each normalized brain MRI image in the training data set into multiple units of size 60x60x60 with a step size of 20x20x20; iteratively performing the following process until the training end condition is met to obtain the lesion site. A model for detecting changes in brain lesions of multiple sclerosis: inputting each brain magnetic resonance imaging image of the preprocessed training data set into the model to be trained for detecting changes in brain lesions of multiple sclerosis, obtaining a detection result of the lesion site at each time, and a change detection result of each lesion site at each time based on multiple time ranges, comparing the obtained lesion site detection result at each time and the change detection result of each lesion site at each time based on multiple time ranges with the manually labeled lesion site at each time, calculating the Dice loss, and adjusting the parameters of the model for detecting changes in brain lesions of multiple sclerosis through a gradient descent strategy based on the Dice loss.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention constructs a model for detecting changes in the lesion site of multiple sclerosis in the brain. The model for detecting changes in the lesion site of multiple sclerosis in the brain includes a visual feature encoder, an image registration model, a time-domain Transformer encoder, a multi-layer perceptron and a Transformer decoder. In the detection process, the visual feature is first extracted by the visual feature encoder, and the visual feature is aligned by the image registration model. Then, the brain changes at multiple time points are learned by the time-domain Transformer model. Then, the location of the multiple sclerosis lesion in the brain at each time is detected by the multi-layer perceptron. The change characteristics of each lesion site of the multiple sclerosis patient in the brain based on multiple time ranges at each time are detected by the Transformer decoder, and then the change trend diagram of each lesion site of the multiple sclerosis patient in the brain within multiple time ranges is generated and output, including newly appeared lesions, enlarged or reduced lesions, so that the changes of the multiple sclerosis in the brain over a long period of time can be effectively tracked, and doctors can be better assisted in tracking, analyzing and diagnosing the disease.

[0013] 2. The visual feature encoder of the model for detecting changes in the lesion site of multiple sclerosis in the brain of the present invention includes an imaging genomics model, a deep convolutional network model, and a visual Transformer encoder. Therefore, the visual feature encoder can effectively combine underlying information, high-level semantic features, and relational features, thereby effectively obtaining more accurate lesion site change detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 The figure is a schematic diagram of the flow of the method for detecting changes in the lesion site of multiple sclerosis in the brain according to the present invention. DETAILED DESCRIPTION

[0015] The present invention will be described in detail below in conjunction with the embodiments and drawings. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0016] refer to Figure 1 A method for detecting changes in the lesion site of multiple sclerosis in the brain comprises the following steps: Step 1, obtaining multiple time periods of a patient with multiple sclerosis in the brain t MRI of the brain L t , for each brain MRI image L t Perform preprocessing.

[0017] Step 2: construct a model for detecting changes in the site of multiple sclerosis lesions in the brain and train it. The model for detecting changes in the site of multiple sclerosis lesions in the brain includes a visual feature encoder, an image registration model, a time-domain Transformer encoder, a multi-layer perceptron and a Transformer decoder. The output end of the visual feature encoder is connected to the image registration model, the output end of the image registration model is connected to the time-domain Transformer encoder and the multi-layer perceptron, and the output end of the time-domain Transformer encoder is connected to the multi-layer perceptron and the Transformer decoder.

[0018] Step 3: Each brain MRI image after preprocessing L t The data are input into the model for detecting the changes of the lesion sites of multiple sclerosis in the brain to detect the changes of the lesion sites of the multiple sclerosis patients in the brain, and the change characteristics of each lesion site of the multiple sclerosis patients in the brain based on multiple time ranges at each time are obtained, and the lesion location detection map of the multiple sclerosis patients in the brain at each time is obtained. M t , this changing feature can reflect the evolution of the disease.

[0019] Step 4: Based on the change characteristics of each lesion site in the brain of the multiple sclerosis patient obtained in step 3 at each time and based on multiple time ranges, generate and output a change trend graph of each lesion site in the brain of the multiple sclerosis patient within multiple time ranges.

[0020] The changing trend chart of each lesion site in the brain of patients with multiple sclerosis within multiple time ranges can reflect the changes in the number of brain lesions, the changes in the size of each lesion, and the emergence of new lesions in patients at different times.

[0021] Furthermore, the step 3 is specifically as follows: 3.1 Multiple time visual feature encoding: each preprocessed brain MRI image L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t .

[0022] 3.2 Alignment of visual features at multiple times: Align each visual feature map F t Input to the image registration model to obtain the aligned visual feature map at each time F t '.

[0023] 3.3 Temporal feature extraction of multiple times: Align each visual feature map at each time F t 'All are input into the time domain Transformer encoder to obtain the time domain feature map at each time F t ^, time domain feature map at each time F t ^Both include features of brain changes in multiple sclerosis patients over multiple time frames.

[0024] 3.4 Detection of multiple sclerosis lesions in the brain: Visual feature maps at each time F t 'And its corresponding time domain feature map F t ^After cascading, they are all input into the multi-layer perceptron to obtain the lesion location detection map of the brain of multiple sclerosis patients at each time. M t .

[0025] The time domain features contain the information about the evolution of the disease in the brain of patients with multiple sclerosis in multiple time ranges. The time domain features can assist the visual features. Combining the time domain features with the visual features can generate a more accurate lesion location detection map of patients with multiple sclerosis at each time.M t .

[0026] 3.5 Change detection of multiple sclerosis lesion sites in the brain at multiple times: Visual feature maps at each time F t 'And the time domain feature map corresponding to the time F t ^ are input into the Transformer decoder to obtain the change characteristics of each lesion site in the brain of patients with multiple sclerosis based on multiple time ranges at each time. In the spatiotemporal attention module of the Transformer decoder, the query is the visual feature map at each time. F t ', Key is the time domain feature map for each time F t ^, Value is the time domain feature map for each time F t ^, Transformer decoder can learn the changes of each area of ​​the brain at multiple time points, and obtain the change characteristics of each lesion site in the brain of patients with multiple sclerosis based on multiple time ranges at each time.

[0027] Detection map of lesion location at each time in the brain of patients with multiple sclerosis M t The changing trend diagram of each lesion site in patients with multiple sclerosis of the brain based on multiple time ranges at each time can effectively assist the diagnosis and clinical treatment of multiple sclerosis of the brain.

[0028] Preferably, the above 3.1 processes each preprocessed brain magnetic resonance imaging L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t , specifically as follows: the visual feature encoder includes an imaging omics model, a deep convolutional network model, and a visual Transformer encoder. The output ends of the imaging omics model and the deep convolutional network model are both connected to the visual Transformer encoder, and the deep convolutional network model adopts a ResNet-50 network model.

[0029] 3.1.1 Each brain MRI image after preprocessing L t They were input into the imaging omics model and the deep convolutional network model respectively.

[0030] 3.1.2 The radiomics model extracts each MRI image after preprocessingL t Radiomics features to obtain each MRI image L t The underlying information.

[0031] 3.1.3 The deep convolutional network model encodes each preprocessed MRI image L t Convolutional features to extract each MRI image L t high-level semantic features.

[0032] 3.1.4 Each MRI image L t The imaging features and convolutional features of the cascaded image are respectively concatenated. L t The imaging features of each MRI image are then fused with the convolutional features to obtain L t The features of the image omics features and convolutional features are integrated.

[0033] 3.1.5 Each MRI image L t The features that integrate radiomics features and convolutional features are input into the visual Transformer encoder for encoding to obtain a visual feature map that integrates radiomics features, convolutional features and relational features at each time. F t .

[0034] The imaging omics model focuses on encoding low-level information, such as brain texture, grayscale, etc., the deep convolutional network model focuses on encoding high-level semantic features, such as brain region location, lesion site features, etc. The visual Transformer encoder focuses on encoding relational features, such as the relationship between different brain regions, the relationship between different lesion sites, the relationship between normal sites and lesion sites, etc. Therefore, the visual feature encoder can effectively combine the above features to obtain more accurate lesion site change detection results.

[0035] Preferably, the visual Transformer encoder includes two encoding layers, the time domain Transformer encoder includes two encoding layers, and the Transformer decoder includes two decoding layers, which improves computing efficiency while effectively extracting features.

[0036] Each encoding layer of the visual Transformer encoder includes a spatial self-attention module and a linear layer, and each decoding layer of the Transformer decoder includes a spatiotemporal attention module, a spatial self-attention module and a linear layer.

[0037] Optionally, each brain magnetic resonance imaging L t The preprocessing process includes the following steps: 1.1 For each brain MRI image L t Perform normalization.

[0038] 1.2 Each brain MRI image after normalization L t Cropped into multiple cells of size 60x60x60 with a stride of 20x20x20 to detect small lesions.

[0039] Optionally, the step 3.2 is as follows: first, each brain magnetic resonance imaging image after preprocessing is L t All of them are input into the image registration model, and the co-registration method is used in the image registration model to register multiple pre-processed brain magnetic resonance images. L t Alignment can establish a coordinate system to align each preprocessed MRI image L t The points representing the same position in the brain are placed at the same position in the coordinate system, and then the visual feature map at each time F t are input into the image registration model based on multiple preprocessed brain MRI images. L t The alignment result of each time visual feature map is used in the image registration model. F t Align to obtain the aligned visual feature map at each time F t '.

[0040] Preferably, the training of the model for detecting changes in the site of multiple sclerosis lesions in the brain is specifically as follows: A training data set is obtained, wherein the training data set includes brain magnetic resonance imaging of multiple sclerosis patients at multiple times and manually annotated lesion sites at each time.

[0041] Each brain MRI image in the training data set is preprocessed as follows: each brain MRI image in the training data set is normalized; each normalized brain MRI image in the training data set is cropped into multiple units of size 60x60x60 with a step size of 20x20x20 to detect small lesions.

[0042] The following process is iteratively performed until the training end condition is met to obtain a model for detecting changes in brain multiple sclerosis lesion sites: each brain magnetic resonance image of the preprocessed training data set is input into the model to be trained for detecting changes in brain multiple sclerosis lesion sites, and the lesion site detection result at each time and the change detection result of each lesion site at each time based on multiple time ranges are obtained, and the obtained lesion site detection result at each time and the change detection result of each lesion site at each time based on multiple time ranges are compared with the manually labeled lesion site at each time, and the Dice loss is calculated. Based on the Dice loss, the model for detecting changes in brain multiple sclerosis lesion sites is parameterized by a gradient descent strategy, and the learning rate is set to 0.01 during parameter adjustment.

[0043] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for detecting changes in brain multiple sclerosis lesion sites, characterized in that: The following steps are involved: Step 1: Obtain multiple time periods of the brain of patients with multiple sclerosis t MRI of the brain L t , for each brain MRI image L t Perform pre-processing; Step 2, constructing a model for detecting changes in the lesion site of multiple sclerosis in the brain and training it, wherein the model for detecting changes in the lesion site of multiple sclerosis in the brain includes a visual feature encoder, an image registration model, a time-domain Transformer encoder, a multi-layer perceptron and a Transformer decoder, wherein the output end of the visual feature encoder is connected to the image registration model, the output end of the image registration model is connected to the time-domain Transformer encoder and the multi-layer perceptron, and the output end of the time-domain Transformer encoder is connected to the multi-layer perceptron and the Transformer decoder; Step 3: Each brain MRI image after preprocessing L t All of them are input into the model for detecting changes in the lesion site of multiple sclerosis in the brain to detect changes in the lesion site of multiple sclerosis in the brain, and the change characteristics of each lesion site of the multiple sclerosis patient in the brain based on multiple time ranges at each time are obtained, and the lesion location detection map of the multiple sclerosis patient in the brain at each time is obtained. M t ; Step 4: Based on the change characteristics of each lesion site in the brain of the multiple sclerosis patient obtained in step 3 at each time and based on multiple time ranges, generate and output a change trend graph of each lesion site in the brain of the multiple sclerosis patient within multiple time ranges.

2. A method for detecting changes in brain multiple sclerosis lesion sites as claimed in claim 1, characterized in that: The step 3 is specifically as follows: 3.1 Visual feature encoding at multiple times: Each brain MRI image after preprocessing L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t ; 3.2 Alignment of visual features at multiple times: Align each visual feature map F t Input to the image registration model to obtain the aligned visual feature map at each time F t '; 3.3 Temporal feature extraction of multiple times: Align each visual feature map at each time F t 'All are input into the time domain Transformer encoder to obtain the time domain feature map at each time F t ^; 3.4 Detection of multiple sclerosis lesions in the brain: Visual feature maps at each time F t 'And the time domain feature map corresponding to the time F t ^All are cascaded and input into the multi-layer perceptron to obtain the lesion location detection map of the brain of multiple sclerosis patients at each time M t ; 3.5 Change detection of multiple sclerosis lesion sites in the brain at multiple times: Visual feature maps at each time F t 'And the time domain feature map corresponding to the time F t ^are respectively input into the Transformer decoder to obtain the change characteristics of each lesion site in the brain of patients with multiple sclerosis based on multiple time ranges at each time.

3. A method for detecting changes in the lesion site of multiple sclerosis in the brain as claimed in claim 2, characterized in that said step 3.1 is to process each pre-processed brain magnetic resonance image L t Input to the visual feature encoder to get each brain MRI image L t Visual feature map of F t , as follows: The visual feature encoder includes an imaging omics model, a deep convolutional network model, and a visual Transformer encoder, and the output ends of the imaging omics model and the deep convolutional network model are connected to the visual Transformer encoder; 3.1.1 Each brain MRI image after preprocessing L t They were input into the radiomics model and the deep convolutional network model respectively; 3.1.2 The radiomics model extracts each MRI image after preprocessing L t Radiomics features to obtain each MRI image L t The underlying information; 3.1.3 The deep convolutional network model encodes each preprocessed MRI image L t Convolutional features to extract each MRI image L t High-level semantic features of 3.1.4 Each MRI image L t The radiomics features and convolutional features of each MRI image are cascaded and fused. L t The features of the fusion of radiomics features and convolutional features; 3.1.5 Each MRI image L t The features that integrate radiomics features and convolutional features are input into the visual Transformer encoder for encoding to obtain a visual feature map that integrates radiomics features, convolutional features and relational features at each time. F t .

4. A method for detecting changes in brain multiple sclerosis lesion sites as claimed in claim 3, characterized in that: The visual Transformer encoder includes two encoding layers, the time domain Transformer encoder includes two encoding layers, and the Transformer decoder includes two decoding layers.

5. A method for detecting changes in brain multiple sclerosis lesion sites as claimed in claim 4, characterized in that: For each brain MRI image L t The preprocessing steps include: 1.1 For each brain MRI image L t Perform normalization processing; 1.2 Each brain MRI image after normalization L t Cropped into cells of size 60x60x60 with a stride of 20x20x20.

6. A method for detecting changes in brain multiple sclerosis lesion sites as claimed in claim 5, characterized in that: The step 3.2 is as follows: First, each brain MRI image after preprocessing L t are input into the image registration model, and the co-registration method is used in the image registration model to register each preprocessed brain MRI image L t Align, and then map the visual features at each time F t are input into the image registration model, based on each brain MRI image after preprocessing L t The alignment result of each time visual feature map is used in the image registration model. F t Align to obtain the aligned visual feature map at each time F t '.

7. A method for detecting changes in brain multiple sclerosis lesion sites as claimed in claim 1, characterized in that: The training of the model for detecting changes in the location of multiple sclerosis lesions in the brain is as follows: Acquire a training data set, wherein the training data set includes brain magnetic resonance imaging of multiple sclerosis patients at multiple times and manually annotated lesion sites at each time; Each brain MRI image in the training data set is preprocessed as follows: each brain MRI image in the training data set is normalized; each normalized brain MRI image in the training data set is cropped into multiple units of size 60x60x60 with a step size of 20x20x20; The following process is iteratively performed until the training end condition is met to obtain a model for detecting changes in brain multiple sclerosis lesion sites: each brain magnetic resonance imaging image of the preprocessed training data set is input into the model to be trained for detecting changes in brain multiple sclerosis lesion sites, and the lesion site detection result at each time and the change detection result of each lesion site at each time based on multiple time ranges are obtained, the lesion site detection result at each time and the change detection result of each lesion site at each time based on multiple time ranges are compared with the manually marked lesion site at each time, the Dice loss is calculated, and the parameters of the model for detecting changes in brain multiple sclerosis lesion sites are adjusted by the gradient descent strategy based on the Dice loss.

Images