Brain parenchyma MRI image-based infarction change prediction method and system

Abstract

The invention provides a brain parenchyma MRI image-based infarction change prediction method and system, and relates to the technical field of image recognition, and the method comprises the steps: obtaining a brain parenchyma MRI image, constructing a multi-modal image set, carrying out the screening according to a screening standard, carrying out the artifact recognition, calculating a quality score, carrying out the preprocessing, obtaining standard image data, carrying out the feature extraction, and carrying out the prediction of the infarction change of the brain parenchyma MRI image. Obtaining a brain region segmentation mask; performing feature extraction to obtain multi-modal feature data, constructing a topological relation among different brain regions, extracting high-order semantic features, obtaining brain region feature data, and performing selection to obtain high-quality feature data; constructing a time sequence characteristic matrix, distributing weights, generating residual output, performing prediction through a full-connection neural network, generating an infarction change prediction result, generating a prediction prognosis result, and generating a comprehensive prediction report.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method and system for predicting infarction changes based on brain parenchyma MRI images. Background Art

[0002] Cerebral infarction is a common cerebrovascular disease and one of the main causes of death and disability in adults. Timely and accurate prediction of the changing trend and prognosis of cerebral infarction is crucial for developing personalized diagnosis and treatment plans and improving prognosis.

[0003] However, the occurrence and development of cerebral infarction is a dynamic and complex process involving multiple pathophysiological mechanisms. The clinical manifestations and imaging features are heterogeneous and diverse, which brings challenges to predictive analysis. Traditional cerebral infarction prediction methods mainly rely on doctors' clinical experience and qualitative judgment, lack objective and quantitative evaluation indicators, and are easily affected by subjective factors.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for predicting infarction changes based on brain parenchyma MRI images, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for predicting infarction changes based on brain parenchymal MRI images, comprising:

[0007] Acquire brain parenchymal MRI images of the patient and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, extract features from the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask;

[0008] The standard image data and the brain region segmentation mask are added to a pre-constructed three-dimensional deep residual convolutional neural network, feature extraction is performed through densely connected residual blocks, multi-modal feature data is obtained by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy, topological relationships between different brain regions are constructed based on the multi-modal feature data, and high-order semantic features are extracted to obtain brain region feature data, and the brain region feature data is selected through a feature selection network to obtain high-quality feature data;

[0009] A time series feature matrix is ​​constructed based on the high-quality feature data, and time series modeling is performed in combination with the infarction prediction model. Weights are assigned to each time step through the memory unit in the infarction prediction model. Element-level addition is performed in combination with residual connections to generate residual outputs. The residual outputs are spliced ​​to obtain comprehensive time series features. The comprehensive time series features are predicted through a fully connected neural network to generate infarction change prediction results and are predicted again through a prognosis prediction model based on a random forest algorithm. Predicted prognostic results are generated in combination with patient clinical data and risk factors. The infarction change prediction results and the predicted prognostic results are combined to generate a comprehensive prediction report.

[0010] In an optional embodiment,

[0011] Acquire the patient's brain parenchyma MRI image and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, extract features from the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask, including:

[0012] Acquire a brain parenchyma MRI image of the patient, wherein the MRI image includes a T1-weighted image, a T2-weighted image, a fluid-attenuated inversion recovery sequence image, and a diffusion-weighted image, perform de-identification processing on the brain parenchyma MRI image and construct the multimodal image set;

[0013] For each image in the multimodal image set, metadata information corresponding to the current image is obtained, wherein the metadata information includes image size, resolution, and acquisition parameters, and the images in the multimodal image set are screened according to a preset screening standard, and images that do not meet the screening standard are removed to obtain the first image set;

[0014] For the images in the first image set, the correlation coefficient between consecutive slices is calculated, and the motion field of the image sequence is estimated by the optical flow method to remove motion artifacts, the tissue signal intensity corresponding to each pixel in the image is determined by performing intensity statistical analysis on the image data, and the metal artifacts are identified and removed in combination with the signal intensity threshold, the image is edge detected and the signal mutation area and the discontinuous area are determined, the truncation artifacts are identified in the signal mutation area and the discontinuous area, and the identified truncation artifacts are removed, and the quality score of each image is calculated based on the identified artifacts in combination with the artifact type, area and position, and the quality score corresponding to each image is compared with a preset quality score threshold, and the images with quality scores lower than the quality score threshold are removed to obtain the second image set;

[0015] The second image set is preprocessed, non-brain tissue areas are removed by a head region segmentation algorithm based on morphological operations, image registration is performed by a non-rigid registration algorithm based on mutual information, images of different modalities are registered to a standard space and combined with a histogram equalization and a non-local mean filtering algorithm to improve contrast and signal-to-noise ratio to obtain standard image data, feature extraction and forward reasoning are performed on the standard image data by a multi-scale convolutional neural network to obtain a brain region segmentation probability map, and a brain region segmentation mask is obtained by a thresholding operation.

[0016] In an optional embodiment,

[0017] Based on the identified artifacts, the quality score of each image is calculated by combining the artifact type, area and location as shown in the following formula:

[0018]

[0019] Among them, Q w represents the weighted quality score of the image, λ j represents the weight of the jth important area, n j represents the number of artifacts identified in the jth important region, w ti represents the weight coefficient of the i-th artifact type, t i represents the type coefficient of the i-th artifact, w a Represents the weight coefficient of the artifact area, A represents the nonlinear transformation function of the i-th artifact area. i represents the area of ​​the artifact area, A represents the area of ​​the current image, and w d represents the weight coefficient of the artifact position, represents the nonlinear transformation function of the i-th artifact position, d i It represents the distance from the center of the artifact area to the center of the image, and D represents half of the length of the image diagonal.

[0020] In an optional embodiment,

[0021] The standard image data and the brain region segmentation mask are added to a pre-constructed three-dimensional deep residual convolutional neural network, and feature extraction is performed through densely connected residual blocks. Multimodal feature data is obtained by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy. The topological relationship between different brain regions is constructed based on the multimodal feature data, and high-order semantic features are extracted to obtain brain region feature data. The brain region feature data is selected through a feature selection network to obtain high-quality feature data including:

[0022] Adding the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, wherein the three-dimensional deep residual convolutional neural network includes a plurality of residual blocks and transition layers, and each residual block is connected by a dense connection mechanism;

[0023] For the input standard image data, features are extracted through two independent three-dimensional deep residual convolutional neural networks. Each residual block extracts features through two convolutional layers. The features are reused and gradients are directly transferred through identity mapping and skip connections. The outputs of all residual blocks are spliced ​​as the output of the current residual block in combination with the dense connection mechanism to obtain cross-scale features.

[0024] Based on the cross-scale features, the pooling window and step size are reduced according to a progressive downsampling strategy, the downsampling rate is set by hyperparameters and the reduction speed is modified, the cross-scale features between different modalities are fused by adaptive summation, and the last sampling and splicing operations are combined to align to the same resolution to obtain the multimodal feature data;

[0025] Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network, brain regions are used as nodes, and anatomical connections and functional associations between brain regions are used as edges to construct a brain region topology map, feature representations of brain region nodes are learned through graph convolution operations, and interactive information and dependency relationships between brain regions are determined;

[0026] The brain region feature data are added to a preset feature selection network, and based on the interaction information and the dependency relationship, the gating weight corresponding to each brain region feature data is determined through a gated attention mechanism, and the brain region feature data are sorted based on the gating weights, and the top 20% of the brain region feature data are selected as the high-quality feature data and added to the pre-constructed high-quality feature set.

[0027] In an optional embodiment,

[0028] Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network as shown in the following formula:

[0029]

[0030] Among them, X (l+1) represents the node feature matrix of the l+1th layer, σ represents the activation function, k represents the order of the current neighbor, K represents the highest order of aggregated neighbors, C represents the node degree matrix, Bk represents the kth power of the adjacency matrix B, and X (l) represents the node feature matrix of the lth layer, W (l，k) Represents the weight matrix of the l-th layer corresponding to the k-th order neighbor.

[0031] In an optional embodiment,

[0032] A time series feature matrix is ​​constructed based on the high-quality feature data, and a time series modeling is performed in combination with the infarction prediction model. A weight is assigned to each time step through the memory unit in the infarction prediction model, and residual output is generated by element-level addition in combination with residual connection. The residual output is spliced ​​to obtain a comprehensive time series feature, and the comprehensive time series feature is predicted through a fully connected neural network to generate an infarction change prediction result and predict it again through a prognosis prediction model based on a random forest algorithm, and a predicted prognosis result is generated in combination with the patient's clinical data and risk factors, and the infarction change prediction result and the predicted prognosis result are combined to generate a comprehensive prediction report including:

[0033] Based on the high-quality feature data, for the feature dimension in each time step, a corresponding time series feature matrix is ​​constructed, wherein the rows in the time series feature matrix represent the feature vectors of the current time step, and the columns represent the values ​​of the current high-quality feature data at different time steps;

[0034] Add the time series feature matrix to the infarction prediction model, determine the forgetting information in each time step through the memory unit, generate the corresponding time series output based on the forgetting information, assign the corresponding forgetting weight to the current time step according to the proportion of the forgetting information in the original high-quality feature data, generate the initial output of the current forgetting unit based on the forgetting weight, add the output of the current memory unit to the next layer in combination with the residual mechanism, generate the residual output by element-by-element addition for the initial output of each memory unit, the residual output is the hidden state corresponding to the current time series feature matrix, and the residual output is spliced ​​to obtain the comprehensive time series feature;

[0035] Input the comprehensive time series features into a fully connected neural network, perform matrix multiplication on the fully connected weight matrix in the fully connected neural network and the comprehensive time series features, determine the output of the current fully connected layer in combination with a preset bias vector and an activation function, repeat the calculation and perform output prediction through the activation layer to obtain an initial infarction change prediction result, determine the potential feature space based on the initial infarction change prediction result in combination with a preset generative adversarial network, generate high-quality synthetic samples in combination with a generator network and a discriminator network, add the high-quality synthetic samples to the fully connected neural network and generate corresponding synthetic prediction results and compare them with true labels, dynamically adjust the hyperparameters in the fully connected neural network according to the comparison results, and use the fully connected neural network to predict the comprehensive time series features to obtain the infarction change prediction result;

[0036] Based on the infarction change prediction results, a training set is generated by combining self-service sampling and random feature selection, and multiple decision trees are generated through a prognosis prediction model based on a random forest algorithm. The elements in the training set are added to the decision tree for training to obtain the prognosis prediction model, and the infarction change prediction results are added to the prognosis prediction model. The prediction is combined with the imaging features fused with the current patient's clinical data and risk factors to obtain the predicted prognosis results, and a comprehensive prediction report is generated based on the predicted prognosis results and the infarction change prediction results.

[0037] In an optional embodiment,

[0038] The loss function corresponding to the generator network and the discriminator network is shown in the following formula:

[0039] L G =-E[log(H(G(z)))]+λ·||G(z)-x||1;

[0040] Among them, L G represents the generator loss value, E represents the mathematical expectation, H represents the discriminator network, G represents the generator network, λ represents the regularization term weight, x represents the random noise vector, z represents the random noise vector, and G(z) represents the output result of the generator;

[0041]

[0042] Among them, L H represents the discriminator loss value, μ represents the gradient penalty weight, Represents the gradient of the random noise vector x, H(x) the output of the discriminator.

[0043] A second aspect of an embodiment of the present invention provides a system for predicting changes in infarction based on brain parenchyma MRI images, comprising:

[0044] The first unit is used to obtain MRI images of the patient's brain parenchyma and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, perform feature extraction on the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask;

[0045] The second unit is used to add the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, perform feature extraction through densely connected residual blocks, obtain multimodal feature data by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy, construct a topological relationship between different brain regions based on the multimodal feature data and extract high-order semantic features to obtain brain region feature data, and select the brain region feature data through a feature selection network to obtain high-quality feature data;

[0046] The third unit is used to construct a time series feature matrix based on the high-quality feature data, perform time series modeling in combination with the infarction prediction model, assign weights to each time step through the memory unit in the infarction prediction model, perform element-level addition in combination with the residual connection to generate a residual output, splice the residual output to obtain a comprehensive time series feature, predict the comprehensive time series feature through a fully connected neural network, generate an infarction change prediction result, and predict again through a prognosis prediction model based on a random forest algorithm, generate a predicted prognosis result in combination with the patient's clinical data and risk factors, and generate a comprehensive prediction report by combining the infarction change prediction result and the predicted prognosis result.

[0047] According to a third aspect of the embodiments of the present invention,

[0048] An electronic device is provided, comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0052] According to a fourth aspect of the embodiments of the present invention,

[0053] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0054] In the present invention, high-quality standard image data is obtained through multimodal image fusion, quality control, preprocessing and other steps, and multi-scale deep learning is used to extract brain segmentation features. The multimodal features are mined in combination with a three-dimensional residual network, the topological relationship of the brain regions is constructed, and high-quality features are selected to form an end-to-end feature engineering process. The imaging phenotype of cerebral infarction can be comprehensively and accurately characterized, laying a solid foundation for subsequent predictions. The time dependency of cerebral infarction is learned through memory units and residual connections, the evolution characteristics of lesions at different stages are mined, and comprehensive time series features are generated. The dynamic change trajectory of cerebral infarction from the acute stage to the chronic stage can be described, and the continuity and consistency of the prediction can be improved. Comprehensive modeling is performed through machine learning algorithms such as random forests, which can analyze disease risks from multiple dimensions. Discovering image-clinical correlation patterns, improving the accuracy of prognosis prediction, and realizing accurate prognosis assessment driven by imaging genomics, by generating a comprehensive prediction report, intuitively presenting the patient's cerebral infarction change trend, prognosis prediction results, key imaging features, etc., and providing auxiliary diagnosis opinions in the form of quantitative indicators, can help doctors quickly understand the condition and formulate personalized diagnosis and treatment plans. At the same time, it is also convenient to explain the condition and prognosis to patients, improve the quality of doctor-patient communication, and reflect the application value of artificial intelligence in the medical field. In summary, the present invention can not only assist doctors in objective and accurate diagnosis and prognosis judgment, and improve the efficiency and quality of diagnosis and treatment, but also provide new ideas and methods for imaging genomics research on cerebrovascular diseases, artificial intelligence applications, etc., and has broad application prospects and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Schematic diagram of the flow of a method for predicting infarction changes based on brain parenchymal MRI images according to an embodiment of the present invention;

[0056] Figure 2 It is a structural schematic diagram of an infarction change prediction system based on brain parenchymal MRI images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0059] Figure 1FIG. 4 is a flow chart of a method for predicting infarction changes based on brain parenchymal MRI images according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0060] S1. Obtain MRI images of the patient's brain parenchyma and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, perform feature extraction on the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask;

[0061] The brain parenchyma MRI image refers to the image data of the brain parenchyma obtained by using magnetic resonance imaging (MRI) technology, which is often used to examine and analyze brain structure and diagnose lesions (such as tumors, infarctions, etc.). The metadata information is data that describes the data, and provides relevant information about the MRI image, such as acquisition time, equipment parameters, imaging conditions and patient information, etc., to facilitate data storage, retrieval and processing. The artifact recognition result refers to the detection and recognition of artifacts caused by imaging equipment or movement in MRI images. Artifacts may interfere with image quality and diagnostic results. The brain region segmentation mask refers to the binary image of each brain region (such as cerebral cortex, white matter, gray matter, etc.) extracted by the segmentation algorithm in the MRI image, which is used for subsequent analysis and processing.

[0062] In an optional embodiment,

[0063] Acquire the patient's brain parenchyma MRI image and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, extract features from the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask, including:

[0064] Acquire a brain parenchyma MRI image of the patient, wherein the MRI image includes a T1-weighted image, a T2-weighted image, a fluid-attenuated inversion recovery sequence image, and a diffusion-weighted image, perform de-identification processing on the brain parenchyma MRI image and construct the multimodal image set;

[0065] For each image in the multimodal image set, metadata information corresponding to the current image is obtained, wherein the metadata information includes image size, resolution, and acquisition parameters, and the images in the multimodal image set are screened according to a preset screening standard, and images that do not meet the screening standard are removed to obtain the first image set;

[0066] For the images in the first image set, the correlation coefficient between consecutive slices is calculated, and the motion field of the image sequence is estimated by the optical flow method to remove motion artifacts, the tissue signal intensity corresponding to each pixel in the image is determined by performing intensity statistical analysis on the image data, and the metal artifacts are identified and removed in combination with the signal intensity threshold, the image is edge detected and the signal mutation area and the discontinuous area are determined, the truncation artifacts are identified in the signal mutation area and the discontinuous area, and the identified truncation artifacts are removed, and the quality score of each image is calculated based on the identified artifacts in combination with the artifact type, area and position, and the quality score corresponding to each image is compared with a preset quality score threshold, and the images with quality scores lower than the quality score threshold are removed to obtain the second image set;

[0067] The second image set is preprocessed, non-brain tissue areas are removed by a head region segmentation algorithm based on morphological operations, image registration is performed by a non-rigid registration algorithm based on mutual information, images of different modalities are registered to a standard space and combined with a histogram equalization and a non-local mean filtering algorithm to improve contrast and signal-to-noise ratio to obtain standard image data, feature extraction and forward reasoning are performed on the standard image data by a multi-scale convolutional neural network to obtain a brain region segmentation probability map, and a brain region segmentation mask is obtained by a thresholding operation.

[0068] The optical flow method is an algorithm for tracking pixel movement in an image sequence. It analyzes the movement of an object by calculating the change in pixel intensity of the image, and is used for motion detection or tracking in dynamic images. The intensity statistical analysis is a statistical processing of the signal intensity of each pixel or voxel in the MRI image, calculating indicators such as the mean and variance, and is often used to identify abnormal areas or extract features. The signal mutation area refers to an area in the MRI image where the signal intensity changes significantly, which is usually related to lesions or abnormal tissues and may indicate a lesion or injury site. The truncation artifact refers to an artifact caused by signal truncation, incomplete image acquisition or hardware failure, which affects the quality of MRI images and the accuracy of diagnosis. The head area based on morphological operations The segmentation algorithm is an image processing method that uses mathematical morphological operations (such as dilation, erosion, opening operation, and closing operation) to segment the head area, remove the background and non-brain tissue parts, and extract the boundaries of the head. The non-rigid registration algorithm based on mutual information is used to align images under different shooting conditions or modes, and accurately align different brain structures through non-rigid transformation. The non-local mean filtering algorithm is a noise reduction technology that reduces noise while retaining details in the image by comparing similar blocks in the image rather than local pixels. It is suitable for MRI image noise reduction processing. The brain region segmentation probability map is an image calculated based on the segmentation algorithm. Each pixel has a probability of belonging to a specific brain region, which is used to determine the boundaries and regions of different brain regions.

[0069] Obtain the patient's brain parenchyma MRI images, including T1-weighted images, T2-weighted images, fluid-attenuated inversion recovery sequence images (FLAIR), diffusion-weighted images (DWI) and other modalities, perform de-identification processing, delete the patient's name, ID and other privacy information contained in the image, build a multimodal image data set, traverse each image in the multimodal image set, extract its metadata information, including image size, resolution, acquisition parameters, etc., screen the images according to pre-set screening criteria, such as image size, resolution, acquisition equipment model, etc., remove the images that do not meet the standards, and obtain the first image set;

[0070] For each image sequence in the first image set, the correlation coefficient between consecutive slices is calculated to determine whether there are artifacts caused by patient movement. If so, the motion field of the image sequence is estimated using the optical flow method, and the image is motion corrected to remove motion artifacts. The intensity of the corrected image data is statistically analyzed to calculate the signal intensity of each pixel. Combined with the preset signal intensity threshold, the area with abnormally high signal intensity is identified and determined as a metal artifact area, which is removed. The image is edge detected to locate the signal intensity mutation area and discontinuous area, and the morphological characteristics of these areas are analyzed to identify artifacts that may be caused by truncation and other reasons, and remove them.

[0071] According to the identified artifacts, combined with their type, area, location and other information, the quality score of each image is calculated, the quality score is compared with the preset threshold, and the images with quality scores lower than the threshold are eliminated to obtain the second image set. The head region of the image is segmented by an algorithm based on morphological operations to remove non-brain tissue areas such as the scalp and skull. The images of different modalities are registered to the standard space by a non-rigid registration algorithm based on mutual information to achieve image standardization. The contrast of the registered images is enhanced by a histogram equalization algorithm, and then denoised by a non-local mean filtering algorithm to improve the signal-to-noise ratio to obtain standardized image data. The standardized image data is input into a pre-trained multi-scale convolutional neural network, and the probability of each pixel belonging to a different brain region is obtained by forward reasoning to generate a brain region segmentation probability map. The probability map is thresholded, and pixels with probabilities greater than the threshold are marked as corresponding brain regions to obtain the final brain region segmentation mask.

[0072] For example, a patient named Zhang who was suspected of having Alzheimer's disease was admitted to the neurology department of a hospital. To make a clear diagnosis, an MRI examination was performed on him, and multimodal brain images including T1-weighted images, T2-weighted images, FLAIR images, and DWI images were obtained. These images were de-identified, and the patient's name, ID, and other information contained in the images were deleted. A multimodal image dataset was constructed. For each image in the dataset, metadata information such as image size, resolution, and acquisition parameters were extracted. According to the preset screening criteria, two T2-weighted images whose resolution did not meet the requirements were eliminated, and the first image set was obtained;

[0073] For each sequence in the first image set, the correlation coefficient between consecutive slices was calculated, and it was found that there were 3 areas with abnormally low correlation coefficients in the FLAIR sequence, which were determined to be motion artifacts. The motion field of the FLAIR sequence was estimated using the optical flow method, and motion correction was performed. The intensity statistics of the corrected images showed that the signal intensity of one T1-weighted image in the left temporal lobe area was abnormally increased, which was determined to be a metal artifact and eliminated.

[0074] Edge detection was performed on all images, and a total of 8 signal mutation areas and 5 discontinuity areas were located. After morphological analysis, it was determined that 2 signal mutations and 1 discontinuity were artifacts caused by scan truncation, so they were removed. According to the artifact recognition results, the quality score of each image was calculated. The quality score of 1 DWI image was 0.6, which was lower than the preset threshold of 0.8, so it was removed. Finally, the second image set containing 5 T1-weighted images, 4 T2-weighted images, 3 FLAIR images, and 2 DWI images was obtained. The second image set was preprocessed, and non-brain tissue areas were removed using morphological operations. All images were registered to the MNI152 standard space using a registration algorithm based on mutual information. Histogram equalization was performed on the registered images to enhance the contrast of gray and white matter. Non-local mean filtering was used to denoise the images, and standardized image data was finally obtained.

[0075] The standardized images were input into a pre-trained multi-scale convolutional neural network for inference, and a probability map of each pixel belonging to gray matter, white matter, and cerebrospinal fluid was obtained. The probability map was thresholded and a brain segmentation mask was generated. Quantitative calculations showed that the hippocampal volume of patient Zhang was significantly smaller than that of normal people of the same age. Combined with the clinical symptoms, the doctor finally diagnosed him with Alzheimer's disease.

[0076] In this embodiment, by performing a series of preprocessing operations such as de-identification, quality screening, and artifact removal on multimodal MRI images, the standardization degree of the data is effectively improved, and the problem of unstable segmentation performance caused by data quality differences is reduced. Morphological operations are used to achieve automatic head extraction, and a mutual information-based registration algorithm is used to achieve automatic cross-modal registration, which greatly improves the processing efficiency. Images of different modalities provide complementary biophysical information on the soft tissue structure of the brain. Multimodal fusion can describe brain tissue from different angles, which helps to improve the performance of the segmentation algorithm, especially the segmentation effect of tissues with complex structures and lack of contrast such as gray and white matter. In summary, this embodiment achieves robust and efficient brain region segmentation, and can quantitatively describe the morphological characteristics of brain regions, which provides new ideas and methods for imaging-assisted diagnosis of brain diseases, and is expected to improve the accuracy and efficiency of disease diagnosis, and has broad application prospects.

[0077] In an optional embodiment,

[0078] Based on the identified artifacts, the quality score of each image is calculated by combining the artifact type, area and location as shown in the following formula:

[0079]

[0080] Among them, Q w represents the weighted quality score of the image, λ j represents the weight of the jth important area, nj represents the number of artifacts identified in the jth important region, w ti represents the weight coefficient of the i-th artifact type, t i represents the type coefficient of the i-th artifact, w a Represents the weight coefficient of the artifact area, A represents the nonlinear transformation function of the i-th artifact area. i represents the area of ​​the artifact area, A represents the area of ​​the current image, and w d represents the weight coefficient of the artifact position, represents the nonlinear transformation function of the i-th artifact position, d i It represents the distance from the center of the artifact area to the center of the image, and D represents half of the length of the image diagonal.

[0081] In this embodiment, the quality evaluation formula integrates multiple local factors such as artifact type, area, and position, and can more comprehensively and meticulously characterize the local detail differences in image quality. The quality score is a continuous value, which can easily compare and rank the quality of different images, and facilitate subsequent quality screening and filtering. The weighted strategy makes the quality evaluation pay more attention to areas that are important for disease diagnosis, and is relatively insensitive to artifacts in areas where lesions do not often occur, thereby improving the pertinence of the quality evaluation. Different weight coefficients are introduced for different types of artifacts, and the sensitivity of the quality score to different artifact types can be automatically adjusted, so that the evaluation results are more in line with the subjective perception of the actual image quality. In summary, this embodiment can automatically identify and filter low-quality images, and screen out high-quality, high-correlation high-quality image subsets, so as to provide more reliable data support for subsequent image analysis and disease diagnosis, and improve the accuracy and efficiency of image analysis.

[0082] S2. adding the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, performing feature extraction through densely connected residual blocks, combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy to obtain multimodal feature data, constructing topological relationships between different brain regions based on the multimodal feature data and extracting high-order semantic features to obtain brain region feature data, and selecting the brain region feature data through a feature selection network to obtain high-quality feature data;

[0083] The three-dimensional deep residual convolutional neural network (3D ResNet) is a deep learning model that combines a convolutional neural network and a residual network, and is used to process three-dimensional image data (such as MRI images) and extract high-level features. The dual-path multi-scale feature fusion algorithm uses a dual-path structure and fuses image features at multiple scales to enhance the model's ability to recognize brain regions or lesions of different sizes and shapes, and is suitable for medical image analysis. The progressive downsampling strategy is to gradually reduce the resolution when processing images to reduce the computational burden while retaining key details in high-resolution images. It is often used for segmentation and classification tasks. The brain region feature data refers to the feature information of each brain region extracted from MRI images, such as volume, density, signal intensity, etc., and is usually used for disease detection or research. The feature selection network is a deep learning model that is used to automatically select the most relevant features, reduce input dimensions and improve the prediction performance of the model, especially in large-scale feature spaces.

[0084] In an optional embodiment,

[0085] The standard image data and the brain region segmentation mask are added to a pre-constructed three-dimensional deep residual convolutional neural network, and feature extraction is performed through densely connected residual blocks. Multimodal feature data is obtained by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy. The topological relationship between different brain regions is constructed based on the multimodal feature data, and high-order semantic features are extracted to obtain brain region feature data. The brain region feature data is selected through a feature selection network to obtain high-quality feature data including:

[0086] Adding the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, wherein the three-dimensional deep residual convolutional neural network includes a plurality of residual blocks and transition layers, and each residual block is connected by a dense connection mechanism;

[0087] For the input standard image data, features are extracted through two independent three-dimensional deep residual convolutional neural networks. Each residual block extracts features through two convolutional layers. The features are reused and gradients are directly transferred through identity mapping and skip connections. The outputs of all residual blocks are spliced ​​as the output of the current residual block in combination with the dense connection mechanism to obtain cross-scale features.

[0088] Based on the cross-scale features, the pooling window and step size are reduced according to a progressive downsampling strategy, the downsampling rate is set by hyperparameters and the reduction speed is modified, the cross-scale features between different modalities are fused by adaptive summation, and the last sampling and splicing operations are combined to align to the same resolution to obtain the multimodal feature data;

[0089] Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network, brain regions are used as nodes, and anatomical connections and functional associations between brain regions are used as edges to construct a brain region topology map, feature representations of brain region nodes are learned through graph convolution operations, and interactive information and dependency relationships between brain regions are determined;

[0090] The brain region feature data are added to a preset feature selection network, and based on the interaction information and the dependency relationship, the gating weight corresponding to each brain region feature data is determined through a gated attention mechanism, and the brain region feature data are sorted based on the gating weights, and the top 20% of the brain region feature data are selected as the high-quality feature data and added to the pre-constructed high-quality feature set.

[0091] The identity mapping refers to a mapping function where the input is equal to the output. It is often used in residual network structures. The identity mapping ensures that the network can skip unnecessary layers, thereby alleviating the gradient vanishing problem.

[0092] The standardized multimodal MRI image data and the brain segmentation mask obtained by the convolutional neural network are added to the pre-built three-dimensional deep residual convolutional neural network for feature extraction. The network consists of multiple residual blocks and transition layers. The residual blocks are connected by dense connections to achieve feature reuse and gradient direct transmission. The input standard image data are sent to two independent three-dimensional residual networks for feature extraction. Each residual block contains two convolutional layers, and feature reuse is achieved through identity mapping and jump connections.

[0093] The outputs of all residual blocks are spliced ​​together through dense connections as the output of the current residual block to obtain cross-scale image features. Based on the cross-scale features, the feature map is downsampled through a progressive downsampling strategy. The downsampling rate and reduction speed are controlled by setting hyperparameters, and the features extracted from different modalities are fused using an adaptive summation operation. The feature maps are aligned to the same resolution through upsampling and splicing operations to obtain multi-modal fused feature data.

[0094] The multimodal feature data is input into the graph convolutional neural network to extract high-order semantic features. The brain regions are regarded as nodes of the graph, and the anatomical connections and functional associations between brain regions are used as edges of the graph to construct a brain region topology map. The feature representation of each brain region node is learned through graph convolution operations, and the interactive information and dependency relationships between brain regions are mined. The learned brain region feature data is sent to the preset feature selection network for high-quality feature selection. The gated attention mechanism is used to calculate the importance weight of each brain region feature according to the interactive information and dependency relationships between brain regions. The brain region features are sorted according to the weight size, and the top 20% of the features are selected as high-quality features and added to the preset high-quality feature set.

[0095] For example, a research team collected structural MRI and functional MRI data from 100 healthy volunteers, with the goal of mining key brain regions and their connection patterns related to memory ability through multimodal image fusion and brain network analysis, and providing imaging markers for early warning of memory disorders. The MRI data of all subjects were standardized and preprocessed to obtain standardized T1, DTI, and fMRI images. The T1 images were segmented using a pre-trained multi-scale convolutional neural network to obtain segmentation masks of 90 brain regions. The standardized images and segmentation masks were input into a three-dimensional residual convolutional neural network for feature extraction, which contained 4 residual blocks and 3 transition layers. Each residual block contained 2 convolutional layers. The residual blocks were connected by dense connections. Features were extracted from the images of the three modalities of T1, DTI, and fMRI, respectively, to obtain cross-scale image feature maps.

[0096] The feature maps were downsampled using a progressive downsampling strategy, with the downsampling rate gradually increasing from 1 to 4. The feature maps of the three modalities were fused using an adaptive summation algorithm, and the features were aligned to the same resolution through upsampling to obtain the fused multimodal feature data. A topological map of 90 brain regions was constructed based on the anatomical template, and the multimodal feature data was input into the graph convolution network for graph embedding. The high-order features of each brain region node were extracted through graph convolution operations, and the interaction patterns between brain regions were learned. The learned brain region features were input into the gated attention network for high-quality feature selection. The network automatically calculated the importance weight of each brain region feature based on the interaction intensity and dependency between brain regions, sorted all brain region features from large to small according to the weight, selected the top 20% of the features as high-quality features, and formed a set of key brain region features related to memory ability. Finally, 10 key brain regions closely related to memory ability, including hippocampus, entorhinal cortex, and posterior cingulate gyrus, were identified, and the functional connections and interaction patterns between these brain regions were excavated, laying the foundation for further clinical verification and disease early warning.

[0097] In this embodiment, multimodal fusion can comprehensively utilize the brain structure and function information provided by different physical imaging mechanisms, characterize the characteristics of brain regions from multiple angles, and help improve the accuracy and robustness of subsequent analysis. Low-level residual blocks extract local detail features, and high-level residual blocks extract global abstract features. Features of different scales are spliced ​​in the channel dimension, which can simultaneously utilize local and global feature information to improve the richness and distinctiveness of feature representation. Adaptive fusion can automatically adjust the fusion strategy according to the local pattern and global statistics of the feature map, thereby obtaining a more semantically consistent multimodal feature representation. The brain networking method takes into account the anatomical connection and functional dependence between brain regions, and models the interaction between brain regions. The pattern is used to extract higher-level brain network features that are more in line with cognitive mechanisms, which is helpful to characterize the distributed processing characteristics of brain cognitive functions. The attention mechanism can adaptively adjust feature weights according to task objectives, select high-quality feature subsets that are more discriminative and stable, and improve the pertinence and effectiveness of subsequent analysis. In summary, this embodiment makes full use of the advantages of deep learning in feature extraction and selection, and combines the brain network analysis model to directly learn high-quality and highly relevant key brain region feature representations from original brain images, providing more accurate and richer imaging genomics indicators for brain disease diagnosis, cognitive research, etc., and has broad application prospects in the fields of brain science research and brain disease diagnosis and treatment.

[0098] In an optional embodiment,

[0099] Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network as shown in the following formula:

[0100]

[0101] Among them, X (l+1) represents the node feature matrix of the l+1th layer, σ represents the activation function, k represents the order of the current neighbor, K represents the highest order of aggregated neighbors, C represents the node degree matrix, Bk represents the kth power of the adjacency matrix B, and X (l) represents the node feature matrix of the lth layer, W (l，k) Represents the weight matrix of the l-th layer corresponding to the k-th order neighbor.

[0102] In this embodiment, by aggregating the feature information of neighbors of different orders, the multi-hop connection relationship and long-range dependency between nodes can be captured, the high-order semantic features contained in the brain network can be extracted, and the functional interaction patterns between brain regions can be more comprehensively characterized. By jointly optimizing node features and neighbor weights in an end-to-end training manner, the optimal feature aggregation strategy can be automatically discovered to improve the distinctiveness and robustness of feature representation. Node features can integrate more structural information and semantic information layer by layer, abstract layer by layer, and finally obtain a high-level brain network feature representation. In summary, this embodiment can fully mine the high-level semantic information and interaction patterns contained in the brain network through adaptive learning of neighbor weights and node features, and construct a more accurate and comprehensive brain region feature representation, which provides new ideas and tools for subsequent brain network analysis and brain disease diagnosis, and is expected to promote the in-depth application of artificial intelligence technology in brain science research.

[0103] S3. Construct a time series feature matrix based on the high-quality feature data, perform time series modeling in combination with the infarction prediction model, assign weights to each time step through the memory unit in the infarction prediction model, perform element-level addition in combination with the residual connection to generate a residual output, splice the residual output to obtain a comprehensive time series feature, predict the comprehensive time series feature through a fully connected neural network, generate an infarction change prediction result, and predict again through a prognosis prediction model based on a random forest algorithm, generate a predicted prognosis result in combination with the patient's clinical data and risk factors, and generate a comprehensive prediction report by combining the infarction change prediction result and the predicted prognosis result.

[0104] The time series feature matrix refers to the features of time series data represented in matrix form, where the rows of the matrix represent time points and the columns represent different features, which are used for time series prediction or analysis. The infarction prediction model is a model for predicting the risk of cerebral infarction, which is based on the patient's historical data and imaging features, combined with statistical analysis or machine learning algorithms, to predict the possibility of future infarction. The memory unit refers to an information unit used to store and process long-term dependencies in a recurrent neural network (such as LSTM or GRU), which is used to process time series data. The prognosis prediction model based on the random forest algorithm is used to predict the prognosis of the disease (such as recovery status, occurrence of complications, etc.) based on the patient's characteristics and medical history data. The risk factors refer to characteristics or conditions that increase the probability of disease occurrence, which may include age, living habits, genes, etc.

[0105] In an optional embodiment,

[0106] A time series feature matrix is ​​constructed based on the high-quality feature data, and a time series modeling is performed in combination with the infarction prediction model. A weight is assigned to each time step through the memory unit in the infarction prediction model, and residual output is generated by element-level addition in combination with residual connection. The residual output is spliced ​​to obtain a comprehensive time series feature, and the comprehensive time series feature is predicted through a fully connected neural network to generate an infarction change prediction result and predict it again through a prognosis prediction model based on a random forest algorithm, and a predicted prognosis result is generated in combination with the patient's clinical data and risk factors, and the infarction change prediction result and the predicted prognosis result are combined to generate a comprehensive prediction report including:

[0107] Based on the high-quality feature data, for the feature dimension in each time step, a corresponding time series feature matrix is ​​constructed, wherein the rows in the time series feature matrix represent the feature vectors of the current time step, and the columns represent the values ​​of the current high-quality feature data at different time steps;

[0108] Add the time series feature matrix to the infarction prediction model, determine the forgetting information in each time step through the memory unit, generate the corresponding time series output based on the forgetting information, assign the corresponding forgetting weight to the current time step according to the proportion of the forgetting information in the original high-quality feature data, generate the initial output of the current forgetting unit based on the forgetting weight, add the output of the current memory unit to the next layer in combination with the residual mechanism, generate the residual output by element-by-element addition for the initial output of each memory unit, the residual output is the hidden state corresponding to the current time series feature matrix, and the residual output is spliced ​​to obtain the comprehensive time series feature;

[0109] Input the comprehensive time series features into a fully connected neural network, perform matrix multiplication on the fully connected weight matrix in the fully connected neural network and the comprehensive time series features, determine the output of the current fully connected layer in combination with a preset bias vector and an activation function, repeat the calculation and perform output prediction through the activation layer to obtain an initial infarction change prediction result, determine the potential feature space based on the initial infarction change prediction result in combination with a preset generative adversarial network, generate high-quality synthetic samples in combination with a generator network and a discriminator network, add the high-quality synthetic samples to the fully connected neural network and generate corresponding synthetic prediction results and compare them with true labels, dynamically adjust the hyperparameters in the fully connected neural network according to the comparison results, and use the fully connected neural network to predict the comprehensive time series features to obtain the infarction change prediction result;

[0110] Based on the infarction change prediction results, a training set is generated by combining self-service sampling and random feature selection, and multiple decision trees are generated through a prognosis prediction model based on a random forest algorithm. The elements in the training set are added to the decision tree for training to obtain the prognosis prediction model, and the infarction change prediction results are added to the prognosis prediction model. The prediction is combined with the imaging features fused with the current patient's clinical data and risk factors to obtain the predicted prognosis results, and a comprehensive prediction report is generated based on the predicted prognosis results and the infarction change prediction results.

[0111] The forgotten information refers to the phenomenon that in a recurrent neural network (such as LSTM), the model cannot remember long-term dependencies due to a long time span or information loss during the weight update process. The decision tree is a classification or regression algorithm that recursively constructs a tree structure by dividing the data into multiple subsets for prediction or decision-making. It is often used in the construction of models with strong explanatory power.

[0112] For each time step, the feature vector of the current time step is used as the row of the matrix, and the high-quality feature values ​​of different time steps are used as the columns of the matrix to form a time series feature matrix. The time series feature matrix is ​​input into the memory unit of the infarction prediction model. The forgetting information of each time step is determined through the memory unit, and the corresponding time series output is generated based on the forgetting information. According to the proportion of the forgetting information in the original high-quality features, the forgetting weight is assigned to the current time step, and the initial output of the current forgetting unit is generated based on the forgetting weight. The residual mechanism is used to add the output of the current memory unit to the memory unit of the next layer. For the initial output of each memory unit, the residual output is generated by element-by-element addition as the hidden state corresponding to the current time series feature matrix. The residual outputs of all time steps are spliced ​​together to obtain the comprehensive time series features.

[0113] The fully connected weight matrix is ​​multiplied by the comprehensive time series features through matrix multiplication, and the output of the current fully connected layer is determined by combining the bias vector and the activation function. The full connection calculation is repeated multiple times and passes through the activation layer to obtain the initial infarction change prediction result. Based on the initial prediction result, the potential feature space is determined by the generative adversarial network, and the generator network and the discriminator network are used to generate high-quality synthetic samples. The synthetic samples are added to the fully connected network for prediction, and the synthetic prediction results are compared with the true labels. The hyperparameters of the fully connected network are dynamically adjusted according to the comparison results, and finally the infarction change prediction result is obtained;

[0114] Based on the prediction results of infarction changes, bootstrap sampling and random feature selection were used to generate a training set, and a random forest algorithm was used to build a prognosis prediction model. Multiple decision trees were trained based on the training set to form a random forest prediction model. The prediction results of infarction changes were input into the prognosis prediction model, and the predictions were made in combination with the patient's clinical data, risk factors and other imaging features to obtain the prognosis prediction results. Based on the prediction results of infarction changes and prognosis prediction results, a comprehensive prediction report was generated to provide a reference for clinical diagnosis and treatment.

[0115] For example, a 65-year-old male ischemic stroke patient was admitted to the Department of Neurology of a tertiary hospital. The symptom was right limb weakness when he was admitted to the hospital. After admission, he underwent MRI examination. According to the clinical data records and serial MRI images, high-quality brain region feature data were extracted, and the MRI features of 7 time points (1st, 3rd, 7th, 14th, 30th, 90th, and 180th days) were constructed into a time series feature matrix with a matrix dimension of 7×128. The time series feature matrix was input into an LSTM network with 3 layers and 128 units in each layer. The LSTM memory unit controls memory update through a forget gate, and generates forget weights based on the forgetting ratio of high-quality features at different time steps. According to the input at the current moment and the hidden state and memory state at the previous moment, the hidden state and memory state at the current moment are updated. Residual connections are introduced between LSTM layers, and the output of the previous layer is added element by element to generate a residual output of the current time step with a dimension of 128. The residual outputs of the 7 time steps are spliced ​​in the time dimension to obtain a 1024-dimensional comprehensive time series feature.

[0116] The comprehensive time series features are passed through a fully connected network with 3 hidden layers (1024, 512, and 256 neurons), using the ReLU activation function and the Sigmoid function in the network output layer to generate the initial infarction change probability prediction value. A generative adversarial network is built, including 1 generator (3-layer fully connected) and 1 discriminator (4-layer fully connected). Based on the real samples and the initial prediction values, the generative adversarial network is trained to extract the potential feature space and generate 500 synthetic samples. The synthetic samples are input into the fully connected network and the cross entropy loss is calculated with the real labels. The fully connected network parameters are optimized by back propagation. Finally, the prediction accuracy of the fully connected network on the test set reaches 92%, and the infarction change probability prediction result is obtained.

[0117] 500 samples were sampled from the original data as a new training set by bootstrap sampling, and a sub-feature set was constructed by randomly selecting 30% of the feature dimensions. A random forest model containing 100 decision trees was trained based on the new training set to obtain a prognosis prediction model. The prediction results of the patient's infarction changes were input into the random forest model, and combined with 11 clinical risk factors such as age, gender, smoking history, hypertension, and 5 imaging features such as NIHSS score and Barthel index at admission, the patient's 90-day functional prognosis was predicted, and the mRS score was 2 points, indicating a relatively good prognosis. A comprehensive prediction report for the patient was generated, including: basic information of the patient; clinical data, risk factors, etc.; MRI imaging features at admission; prediction curves of infarction change trends at 7 time points, showing that the infarction volume reached a peak on the third day, then gradually decreased, and dropped to less than 20% on 90 days; the 90-day functional prognosis prediction result was mRS2 points, indicating a relatively good prognosis, and continued standardized treatment and rehabilitation training were recommended.

[0118] In this embodiment, the time series feature representation can characterize the continuous change trajectory of the lesion, extract the implicit time dependency, and provide richer feature information for predictive analysis. Residual learning directly transfers the features of the previous layer to the next layer to achieve feature reuse and optimization, which can not only accelerate convergence but also improve the accuracy of feature representation. By introducing residual connections between LSTM layers, the hidden states of different time steps are added element by element, and the residual outputs of all time steps are spliced ​​to form a comprehensive time series feature. Local and global time series information can be aggregated at different time scales to construct a more accurate and comprehensive feature representation of cerebral infarction evolution, laying the foundation for subsequent predictive analysis. The random forest model can automatically discover key imaging indicators and clinical risk factors that affect prognosis, and The prediction results of each decision tree are integrated through the voting mechanism, the deviation of individual decision trees is reduced, and the reliability of the prediction is improved. In the process of prognosis prediction, the prediction results of infarction changes are integrated with the patient's clinical data, risk factors and other imaging characteristics, realizing the organic combination of imaging phenotype and clinical phenotype. It can more comprehensively characterize the patient's condition characteristics from different dimensions, explore the imaging-clinical correlation rules, and provide richer information support for personalized prognosis prediction. In summary, this embodiment can intelligently extract the dynamic evolution characteristics of cerebral infarction, characterize important imaging-clinical correlation patterns, and mine individualized diagnosis and prognosis prediction indicators from multimodal data. It is intuitively presented in the form of a comprehensive report, which provides strong support for precise diagnosis and treatment and clinical decision-making, and has broad application prospects.

[0119] In an optional embodiment,

[0120] The loss function corresponding to the generator network and the discriminator network is shown in the following formula:

[0121] L G=-E[log(H(G(z)))]+λ·||G(z)-x||1;

[0122] Among them, L G represents the generator loss value, E represents the mathematical expectation, H represents the discriminator network, G represents the generator network, λ represents the regularization term weight, x represents the random noise vector, z represents the random noise vector, and G(z) represents the output result of the generator;

[0123]

[0124] Among them, L H represents the discriminator loss value, μ represents the gradient penalty weight, Represents the gradient of the random noise vector x, H(x) the output of the discriminator.

[0125] In this embodiment, the soft constraint method can effectively enhance the stability and generalization of the discriminator and improve the authenticity of the generated samples. The regularization weights can ensure the richness of the generated samples while improving the generation quality and avoiding mode collapse. By training the generator to map the noise to the real sample space, the intrinsic structure and properties of the data distribution can be discovered, and the implicit feature representation can be mined, which is of great value for tasks such as data enhancement and anomaly detection. By generating realistic synthetic samples, the overfitting problem can be alleviated and the prediction performance can be improved. In summary, this embodiment can effectively improve the instability and gradient vanishing problems in the training of the generative adversarial network. By penalizing the gradient norm soft-constrained discriminator, the generation quality and diversity are balanced, the latent space features are mined, and an end-to-end optimization framework is formed with the downstream prediction model, thereby improving the performance and generalization ability of the entire cerebral infarction intelligent prediction system, and playing an important role in data enhancement, feature extraction, model optimization, etc.

[0126] Figure 2 FIG. 4 is a schematic diagram of the structure of the infarction change prediction system based on brain parenchymal MRI images according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0127] The first unit is used to obtain MRI images of the patient's brain parenchyma and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, perform feature extraction on the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask;

[0128] The second unit is used to add the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, perform feature extraction through densely connected residual blocks, obtain multimodal feature data by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy, construct a topological relationship between different brain regions based on the multimodal feature data and extract high-order semantic features to obtain brain region feature data, and select the brain region feature data through a feature selection network to obtain high-quality feature data;

[0129] The third unit is used to construct a time series feature matrix based on the high-quality feature data, perform time series modeling in combination with the infarction prediction model, assign weights to each time step through the memory unit in the infarction prediction model, perform element-level addition in combination with the residual connection to generate a residual output, splice the residual output to obtain a comprehensive time series feature, predict the comprehensive time series feature through a fully connected neural network, generate an infarction change prediction result, and predict again through a prognosis prediction model based on a random forest algorithm, generate a predicted prognosis result in combination with the patient's clinical data and risk factors, and generate a comprehensive prediction report by combining the infarction change prediction result and the predicted prognosis result.

[0130] According to a third aspect of the embodiments of the present invention,

[0131] An electronic device is provided, comprising:

[0132] processor;

[0133] a memory for storing processor-executable instructions;

[0134] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0135] According to a fourth aspect of the embodiments of the present invention,

[0136] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0137] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting infarction changes based on brain parenchymal MRI images, characterized in that: include: Acquire brain parenchymal MRI images of the patient and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, extract features from the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask; The standard image data and the brain region segmentation mask are added to a pre-constructed three-dimensional deep residual convolutional neural network, feature extraction is performed through densely connected residual blocks, multi-modal feature data is obtained by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy, topological relationships between different brain regions are constructed based on the multi-modal feature data, and high-order semantic features are extracted to obtain brain region feature data, and the brain region feature data is selected through a feature selection network to obtain high-quality feature data; A time series feature matrix is ​​constructed based on the high-quality feature data, and time series modeling is performed in combination with the infarction prediction model. Weights are assigned to each time step through the memory unit in the infarction prediction model. Element-level addition is performed in combination with residual connections to generate residual outputs. The residual outputs are spliced ​​to obtain comprehensive time series features. The comprehensive time series features are predicted through a fully connected neural network to generate infarction change prediction results and are predicted again through a prognosis prediction model based on a random forest algorithm. Predicted prognostic results are generated in combination with patient clinical data and risk factors. The infarction change prediction results and the predicted prognostic results are combined to generate a comprehensive prediction report.

2. The method according to claim 1, characterized in that Acquire the patient's brain parenchyma MRI image and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, extract features from the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask, including: Acquire a brain parenchyma MRI image of the patient, wherein the MRI image includes a T1-weighted image, a T2-weighted image, a fluid-attenuated inversion recovery sequence image, and a diffusion-weighted image, perform de-identification processing on the brain parenchyma MRI image and construct the multimodal image set; For each image in the multimodal image set, metadata information corresponding to the current image is obtained, wherein the metadata information includes image size, resolution, and acquisition parameters, and the images in the multimodal image set are screened according to a preset screening standard, and images that do not meet the screening standard are removed to obtain the first image set; For the images in the first image set, the correlation coefficient between consecutive slices is calculated, and the motion field of the image sequence is estimated by the optical flow method to remove motion artifacts, the tissue signal intensity corresponding to each pixel in the image is determined by performing intensity statistical analysis on the image data, and the metal artifacts are identified and removed in combination with the signal intensity threshold, the image is edge detected and the signal mutation area and the discontinuous area are determined, the truncation artifacts are identified in the signal mutation area and the discontinuous area, and the identified truncation artifacts are removed, and the quality score of each image is calculated based on the identified artifacts in combination with the artifact type, area and position, and the quality score corresponding to each image is compared with a preset quality score threshold, and the images with quality scores lower than the quality score threshold are removed to obtain the second image set; The second image set is preprocessed, non-brain tissue areas are removed by a head region segmentation algorithm based on morphological operations, image registration is performed by a non-rigid registration algorithm based on mutual information, images of different modalities are registered to a standard space and combined with a histogram equalization and a non-local mean filtering algorithm to improve contrast and signal-to-noise ratio to obtain standard image data, feature extraction and forward reasoning are performed on the standard image data by a multi-scale convolutional neural network to obtain a brain region segmentation probability map, and a brain region segmentation mask is obtained by a thresholding operation.

3. The method according to claim 2, characterized in that Based on the identified artifacts, the quality score of each image is calculated by combining the artifact type, area and location as shown in the following formula: Among them, Q w represents the weighted quality score of the image, λ j represents the weight of the jth important area, n j represents the number of artifacts identified in the jth important region, w ti represents the weight coefficient of the i-th artifact type, t i represents the type coefficient of the i-th artifact, w a Represents the weight coefficient of the artifact area, A represents the nonlinear transformation function of the i-th artifact area. i represents the area of ​​the artifact area, A represents the area of ​​the current image, and w d represents the weight coefficient of the artifact position, represents the nonlinear transformation function of the i-th artifact position, d i It represents the distance from the center of the artifact area to the center of the image, and D represents half of the length of the image diagonal.

4. The method according to claim 1, characterized in that: The standard image data and the brain region segmentation mask are added to a pre-constructed three-dimensional deep residual convolutional neural network, and feature extraction is performed through densely connected residual blocks. Multimodal feature data is obtained by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy. The topological relationship between different brain regions is constructed based on the multimodal feature data, and high-order semantic features are extracted to obtain brain region feature data. The brain region feature data is selected through a feature selection network to obtain high-quality feature data including: Adding the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, wherein the three-dimensional deep residual convolutional neural network includes a plurality of residual blocks and transition layers, and each residual block is connected by a dense connection mechanism; For the input standard image data, features are extracted through two independent three-dimensional deep residual convolutional neural networks. Each residual block extracts features through two convolutional layers. The features are reused and gradients are directly transferred through identity mapping and skip connections. The outputs of all residual blocks are spliced ​​as the output of the current residual block in combination with the dense connection mechanism to obtain cross-scale features. Based on the cross-scale features, the pooling window and step size are reduced according to a progressive downsampling strategy, the downsampling rate is set by hyperparameters and the reduction speed is modified, the cross-scale features between different modalities are fused by adaptive summation, and the last sampling and splicing operations are combined to align to the same resolution to obtain the multimodal feature data; Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network, brain regions are used as nodes, and anatomical connections and functional associations between brain regions are used as edges to construct a brain region topology map, feature representations of brain region nodes are learned through graph convolution operations, and interactive information and dependency relationships between brain regions are determined; The brain region feature data are added to a preset feature selection network, and based on the interaction information and the dependency relationship, the gating weight corresponding to each brain region feature data is determined through a gated attention mechanism, and the brain region feature data are sorted based on the gating weights, and the top 20% of the brain region feature data are selected as the high-quality feature data and added to the pre-constructed high-quality feature set.

5. The method according to claim 4, characterized in that Based on the multimodal feature data, high-order semantic features are extracted through a graph convolutional neural network as shown in the following formula: Among them, X (l+1) represents the node feature matrix of the l+1th layer, σ represents the activation function, k represents the order of the current neighbor, K represents the highest order of aggregated neighbors, C represents the node degree matrix, Bk represents the kth power of the adjacency matrix B, and X (l) represents the node feature matrix of the lth layer, W (l，k) Represents the weight matrix of the l-th layer corresponding to the k-th order neighbor.

6. The method according to claim 1, characterized in that A time series feature matrix is ​​constructed based on the high-quality feature data, and a time series modeling is performed in combination with the infarction prediction model. A weight is assigned to each time step through the memory unit in the infarction prediction model, and residual output is generated by element-level addition in combination with residual connection. The residual output is spliced ​​to obtain a comprehensive time series feature, and the comprehensive time series feature is predicted through a fully connected neural network to generate an infarction change prediction result and predict it again through a prognosis prediction model based on a random forest algorithm, and a predicted prognosis result is generated in combination with the patient's clinical data and risk factors, and the infarction change prediction result and the predicted prognosis result are combined to generate a comprehensive prediction report including: Based on the high-quality feature data, for the feature dimension in each time step, a corresponding time series feature matrix is ​​constructed, wherein the rows in the time series feature matrix represent the feature vectors of the current time step, and the columns represent the values ​​of the current high-quality feature data at different time steps; Add the time series feature matrix to the infarction prediction model, determine the forgetting information in each time step through the memory unit, generate the corresponding time series output based on the forgetting information, assign the corresponding forgetting weight to the current time step according to the proportion of the forgetting information in the original high-quality feature data, generate the initial output of the current forgetting unit based on the forgetting weight, add the output of the current memory unit to the next layer in combination with the residual mechanism, generate the residual output by element-by-element addition for the initial output of each memory unit, the residual output is the hidden state corresponding to the current time series feature matrix, and the residual output is spliced ​​to obtain the comprehensive time series feature; Input the comprehensive time series features into a fully connected neural network, perform matrix multiplication on the fully connected weight matrix in the fully connected neural network and the comprehensive time series features, determine the output of the current fully connected layer in combination with a preset bias vector and an activation function, repeat the calculation and perform output prediction through the activation layer to obtain an initial infarction change prediction result, determine the potential feature space based on the initial infarction change prediction result in combination with a preset generative adversarial network, generate high-quality synthetic samples in combination with a generator network and a discriminator network, add the high-quality synthetic samples to the fully connected neural network and generate corresponding synthetic prediction results and compare them with true labels, dynamically adjust the hyperparameters in the fully connected neural network according to the comparison results, and use the fully connected neural network to predict the comprehensive time series features to obtain the infarction change prediction result; Based on the infarction change prediction results, a training set is generated by combining self-service sampling and random feature selection, and multiple decision trees are generated through a prognosis prediction model based on a random forest algorithm. The elements in the training set are added to the decision tree for training to obtain the prognosis prediction model, and the infarction change prediction results are added to the prognosis prediction model. The prediction is combined with the imaging features fused with the current patient's clinical data and risk factors to obtain the predicted prognosis results, and a comprehensive prediction report is generated based on the predicted prognosis results and the infarction change prediction results.

7. The method according to claim 6, characterized in that The loss function corresponding to the generator network and the discriminator network is shown in the following formula: L G =-E[log(H(G(z)))]+λ·||G(z)-x||1; Among them, L G represents the generator loss value, E represents the mathematical expectation, H represents the discriminator network, G represents the generator network, λ represents the regularization term weight, x represents the random noise vector, z represents the random noise vector, and G(z) represents the output result of the generator; Among them, L H represents the discriminator loss value, μ represents the weight of the gradient penalty term, ▽x represents the gradient of the random noise vector x, and H(x) is the output result of the discriminator.

8. A system for predicting infarction changes based on brain parenchyma MRI images, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain MRI images of the patient's brain parenchyma and construct a multimodal image set, obtain metadata information for each image in the multimodal image set and filter according to a preset filtering standard to obtain a first image set, perform artifact recognition on the first image set, calculate the quality score of each image according to the artifact recognition result, filter in combination with a preset score threshold to generate a second image set, preprocess the second image set to obtain standard image data, perform feature extraction on the standard image data through multi-scale deep learning, and obtain a brain region segmentation mask; The second unit is used to add the standard image data and the brain region segmentation mask to a pre-constructed three-dimensional deep residual convolutional neural network, perform feature extraction through densely connected residual blocks, obtain multimodal feature data by combining a dual-path multi-scale feature fusion algorithm and a progressive downsampling strategy, construct a topological relationship between different brain regions based on the multimodal feature data and extract high-order semantic features to obtain brain region feature data, and select the brain region feature data through a feature selection network to obtain high-quality feature data; The third unit is used to construct a time series feature matrix based on the high-quality feature data, perform time series modeling in combination with the infarction prediction model, assign weights to each time step through the memory unit in the infarction prediction model, perform element-level addition in combination with the residual connection to generate a residual output, splice the residual output to obtain a comprehensive time series feature, predict the comprehensive time series feature through a fully connected neural network, generate an infarction change prediction result, and predict again through a prognosis prediction model based on a random forest algorithm, generate a predicted prognosis result in combination with the patient's clinical data and risk factors, and generate a comprehensive prediction report by combining the infarction change prediction result and the predicted prognosis result.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images