A lateral ventricle widening prediction method, device, program product, and medium

Abstract

The present application belongs to the field of brain image processing, and particularly relates to a lateral ventricle widening prediction method, device, program product and medium, the steps of the method comprising: performing three-dimensional segmentation processing on a magnetic resonance image through a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result; the three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model; inputting the three-dimensional segmentation result into a prediction model to obtain a classification result of lateral ventricle adverse prognosis. The present application uses an nnUNet network to perform three-dimensional full-automatic segmentation of the lateral ventricle, and a topological loss (TopoLoss) is used in the segmentation process, and then a classification result of lateral ventricle adverse prognosis is obtained, which improves the continuity and accuracy of three-dimensional segmentation, and solves the problem that traditional two-dimensional segmentation cannot capture the overall anatomical morphology of the lateral ventricle.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of brain image processing, in particular to a lateral ventricle widening prediction method, device, program product and medium. BACKGROUND

[0002] Fetal lateral ventricle widening is an important early indicator of neurodevelopmental abnormalities, and its early detection is of great significance for the prevention and intervention of postnatal neurological diseases. Traditional prenatal examination relies on ultrasound imaging, which has real-time and safety, but has obvious limitations in resolution and deep structure display, especially in the assessment of fetal brain deep structures. However, magnetic resonance imaging (MRI) technology has shown great advantages in fetal brain structure assessment due to its high resolution and multi-sequence characteristics. However, most current researches are still limited to the use of single sequence (such as T2 weighted) and two-dimensional view, which cannot fully capture the three-dimensional morphology and dynamic changes of fetal lateral ventricle.

[0003] In the prior art, two-dimensional segmentation models cannot obtain complete lateral ventricle spatial structure, resulting in inaccurate volume and shape measurement; single sequence characteristics limit the comprehensive characterization of pathological state, especially T1 / T2 sequences have been widely used for clinical rapid imaging, while FLAIR sequence is sensitive to brain edema, but is affected by fetal motion artifacts, and is rarely used in practical applications. Therefore, developing a prediction system that integrates three-dimensional segmentation, multi-sequence fusion and dynamic time series modeling can significantly improve the detection and classification accuracy of lateral ventricle widening, and provide important decision basis for clinical early intervention. SUMMARY

[0004] In order to overcome the above-mentioned defects, the present application realizes three-dimensional segmentation of brain images based on nnUNet architecture, compatible with multiple clinical common sequences, and proposes a lateral ventricle widening prediction method, device, program product and medium based on three-dimensional nnUNet network, to realize accurate reconstruction of fetal lateral ventricle and avoid image distortion caused by too long scanning time.

[0005] Based on the above idea, the technical scheme is proposed:

[0006] The lateral ventricle widening prediction method based on three-dimensional nnUNet network includes the following steps:

[0007] The magnetic resonance image is three-dimensionally segmented by a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result;

[0008] The three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model;

[0009] The three-dimensional segmentation result is input into a prediction model to obtain a classification result of lateral ventricle adverse prognosis.

[0010] Further, the three-dimensional segmentation processing adopts cross-modal feature alignment to realize the alignment of features of multiple modalities in space.

[0011] Further, the cross-modal feature alignment is implemented in the encoder of the three-dimensional nnUNet network, and the features of multiple modalities in the encoder are mapped to the same projection space after convolution dimension reduction, thereby realizing the alignment of features of multiple modalities in space.

[0012] Further, the encoder realizes convolution dimension reduction through a plurality of convolution modules, and each convolution module adopts the steps of convolution processing, normalization processing, ReLU activation, and down-sampling.

[0013] Further, the cross-modal feature alignment is implemented between two adjacent convolution modules in the encoder.

[0014] Further, the cross-modal feature alignment further includes consistency optimization of features of the same anatomical structure between multiple modalities by using a contrast loss.

[0015] Further, the training of the three-dimensional multi-sequence segmentation model by using the topological loss function specifically includes: extracting a three-dimensional normal vector field from the boundary of the three-dimensional segmentation result output by the three-dimensional multi-sequence segmentation model, and processing the angle mutation of adjacent normal vectors in the three-dimensional normal vector field to realize the smooth transition of the predicted result structure.

[0016] Further, the steps further include: resampling the magnetic resonance image in a plurality of directions at the same resolution to obtain resampled data, and inputting the resampled data into the three-dimensional nnUNet network for three-dimensional segmentation processing.

[0017] Further, the steps further include the step of time sequence dynamic modeling optimization: obtaining a dynamic change atlas according to the evolution trend of the ventricle structure at different time periods; inputting the time sequence index in the dynamic change atlas and the three-dimensional segmentation result into the prediction model to obtain a classification result of the lateral ventricle adverse prognosis based on dynamic time sequence.

[0018] Further, the steps further include the step of prognosis risk prediction, specifically including:

[0019] inputting the classification result of the lateral ventricle widening degree into a prognosis risk prediction model to output a prognosis classification level; the prognosis risk prediction model includes at least two classifiers, the classifiers are connected in parallel, and the results output by each classifier are weighted and summed and then output.

[0020] Based on the same concept, a lateral ventricle widening prediction device based on a three-dimensional nnUNet network is also proposed, comprising:

[0021] The acquisition module is configured to acquire a magnetic resonance image of the brain.

[0022] The segmentation module is configured to perform three-dimensional segmentation processing on the magnetic resonance image by using a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result.

[0023] The prediction module is configured to input the three-dimensional segmentation result into a prediction model to obtain a classification result of the lateral ventricle widening degree.

[0024] Further, the system further comprises a cross-modal feature alignment module embedded in the segmentation module, which is configured to align features of multiple modalities in space.

[0025] Further, the system further comprises a time sequence dynamic modeling module configured to output a dynamic change atlas according to an evolution trend of the ventricle structure at different gestational weeks.

[0026] Further, the system further comprises a prognosis risk prediction module configured to input the classification result of the lateral ventricle widening degree into a prognosis risk prediction model to output a prognosis classification level.

[0027] Based on the same concept, a program product for lateral ventricle widening prediction based on a three-dimensional nnUNet network is also proposed.

[0028] Based on the same concept, a medium having instructions executable by a processor stored thereon is also proposed.

[0029] Compared with the prior art, the beneficial effects of the present application are as follows: the nnUNet network is used for three-dimensional full-automatic segmentation of the lateral ventricle, and the TopoLoss is used in the segmentation process to obtain a classification result of the lateral ventricle adverse prognosis. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flowchart of the lateral ventricle widening prediction method based on the three-dimensional nnUNet network in Embodiment 1 of the present application;

[0031] Figure 2 A model architecture diagram of the three-dimensional multi-sequence segmentation model in Embodiment 1 of the present application;

[0032] Figure 3 A three-dimensional full-resolution image segmentation network architecture diagram in Embodiment 2 of the present application;

[0033] Figure 4 A flowchart of the prediction process in the prediction model in Embodiment 2 of the present application;

[0034] Figure 5 A step flowchart of the time series dynamic modeling optimization in Embodiment 3 of the present application;

[0035] Figure 6 A structure diagram of the lateral ventricle widening prediction device based on the three-dimensional nnUNet network in Embodiment 5 of the present application;

[0036] Figure 7 The experimental results of risk prediction in Embodiment 5 of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described in detail below in combination with test examples and specific embodiments. However, this should not be understood as limiting the scope of the above-mentioned subject matter of the present application to only the following embodiments. Any technology implemented based on the content of the present application falls within the scope of the present application.

[0038] In the description of the specific embodiments of the present application, the orientation or positional relationship terms such as "up", "down", "left", "right", "center", "inner", "outer", "side", etc. appearing without special indication, are expressions based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product / device / apparatus is usually used. These orientation or positional relationship terms are only for the convenience of describing the present application scheme or simplifying the description in the specific embodiments, for the convenience of the technical personnel to quickly understand the scheme, and are not intended to indicate or imply that a specific device / component / element must have a specific orientation, or be constructed and operated in a specific positional relationship, and therefore cannot be understood as a limitation on the present application.

[0039] In the description of the embodiments of the present application, the technical terms "first", "second", etc. only distinguish one entity or operation from another entity or operation, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is two and more than two, unless otherwise explicitly and specifically limited.

[0040] Reference to "embodiments" herein means that the specific features, structures or properties described in connection with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean that it refers to the same embodiment, nor is it independent or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0041] Embodiment 1

[0042] The lateral ventricle widening prediction method based on the three-dimensional nnUNet network, the flow chart is as shown in Figure 1 The steps include:

[0043] The magnetic resonance image is processed by a three-dimensional multi-sequence segmentation model for three-dimensional segmentation to obtain a three-dimensional segmentation result.

[0044] The three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model. The three-dimensional segmentation result is input into a prediction model to obtain a classification result of lateral ventricle adverse prognosis.

[0045] From the above steps, it can be seen that the method mainly includes three parts: image acquisition, three-dimensional segmentation processing and prediction of lateral ventricle widening degree, and the model architecture of the three-dimensional multi-sequence segmentation model is as shown in Figure 2 The following describes the main three steps:

[0046] 1. Image acquisition

[0047] The magnetic resonance sequence images of the brain are collected, and T1 / T2 fast sequence images are preferentially collected. Multiple other clinical common sequence images are compatible, and image distortion problems caused by too long scanning time are avoided.

[0048] 2. Three-dimensional segmentation processing

[0049] The magnetic resonance sequence images are input into a three-dimensional multi-sequence segmentation model for three-dimensional multi-sequence segmentation processing to obtain a three-dimensional segmentation result, which specifically includes the following contents:

[0050] Firstly, the collected T1 sequence images or T2 sequence images are resampled in sagittal, coronal and transverse directions to form a uniform isotropic resolution (for example, the resolution is 1mm3), and N4 bias field correction (bias field correction based on N4 algorithm), z-score normalization processing (z-score normalization processing, which belongs to intensity normalization processing), brain region ROI cropping (brain region of interest region cropping) are sequentially performed to obtain multi-modal data. The isotropic resolution refers to that the medical image has consistent resolution in three-dimensional space (X, Y, Z axis), that is, the length, width and height of the voxel are equal (such as 1mm*1mm*1mm).

[0051] Secondly, the obtained multi-modal data is input into a three-dimensional multi-sequence segmentation model to output a three-dimensional segmentation result. The three-dimensional multi-sequence segmentation model is constructed based on an nnUNet network architecture, which is a three-dimensional version of 3D U-Net. The three-dimensional multi-sequence segmentation model includes four layers of encoders and symmetrical decoders, each layer adopts a combination of convolution (Conv3D) + normalization + ReLU activation + down-sampling (3D Max Pooling).

[0052] Further, the three-dimensional multi-sequence segmentation processing further includes adding a topological loss function to constrain the topological structure of the three-dimensional multi-sequence segmentation model. When training the three-dimensional multi-sequence segmentation model, a TopologicalLoss function is used to constrain the topological structure of the three-dimensional multi-sequence segmentation model, The three-dimensional normal vector field is extracted from the predicted Mask boundary, the abrupt change of the adjacent normal vector angle is punished, and the structure is smoothly transitioned to ensure that the segmented lateral ventricle region has continuity and integrity in space.

[0053] Punishing the abrupt change of the adjacent normal vector angle mainly refers to: by introducing a loss function TopoLoss, the case of abrupt change of the normal vector angle of adjacent voxels in the segmentation result is numerically punished, that is, in the network training process, the case of abrupt change of the normal vector angle of adjacent voxels will be given a larger loss value, driving the model to reduce such phenomenon. Specifically, in the training process, for the boundary voxel of the segmentation Mask, the normal vector of the three-dimensional boundary is calculated, and for the normal vector of the adjacent voxel, the included angle θ is calculated. If the included angle is large, the loss is large, which promotes the network to learn a more smooth boundary.

[0054] ​Unsmooth means that the mask boundary of segmentation presents jagged, broken, and has sharp corners. The specific implementation of smooth transition of structure is to make the boundary continuous and round, and the normal vector slowly changes in space, and the normal vector of adjacent voxels points to close. The smooth boundary can more truly reflect the integrity and continuity of the anatomical structure, and prevent false positive and false negative areas caused by model prediction noise.

[0055] In the image segmentation task, Mask is a matrix with the same size as the original image, usually a binary graph (0 and 1) or a multi-channel label graph. Its core function is to accurately mark the spatial distribution of the target object: one is a binary classification task: single-channel binary Mask, 1 represents foreground (target), and 0 represents background; the other is multi-class segmentation: each channel corresponds to a binary mask of a class, or uses color labels to distinguish different objects (such as instance segmentation).

[0056] The formula of the topological loss function is shown in formula (1):

[0057] (1)

[0058] wherein, respectively represent the normal vectors between adjacent voxels, i represents the number of one kind of voxels, j represents the number of another kind of voxels, N is the total number of voxels, and voxel refers to volume element, which is the smallest unit of digital data in three-dimensional space segmentation, and is used in the fields of three-dimensional imaging, scientific data and medical imaging.

[0059] 3. Prediction of lateral ventricle widening degree

[0060] The prediction of lateral ventricle widening degree is realized by using a prediction model, and the input of the model comes from the three-dimensional nnUNet segmentation result, and the output is three grades: normal, mild widening, and moderate / severe widening.

[0061] The prediction process is shown in Figure 4 , which includes:

[0062] Based on the three-dimensional segmentation result, the three-dimensional features and multi-view radiomics features of the lateral ventricle and its surrounding white matter region are extracted; the extracted three-dimensional features and multi-view radiomics features are fused to form a high-dimensional feature vector; the high-dimensional feature vector is selected to obtain a feature subset most related to the lateral ventricle widening degree; the feature subset is input into the prediction model, and the classification result of the lateral ventricle widening degree is output. The specific implementation process is as follows:

[0063] 3.1. Three-dimensional feature extraction and multi-modal fusion

[0064] Based on the three-dimensional segmentation result, the stereoscopic features (such as expansion directionality, three-dimensional texture distribution, etc.) and multi-view radiomics features (such as shape features, gray level first-order statistical features, etc.) of the lateral ventricle and its surrounding white matter region are extracted. Specifically, first, the three-dimensional shape principal axis is calculated by PCA principal component analysis, the expansion direction and eccentricity are estimated, and the expansion directionality is determined. Then, the gray level co-occurrence matrix (GLCM) and wavelet decomposition are applied to extract the three-dimensional texture distribution features (such as contrast, energy, uniformity) from each voxel block. Finally, the shape features (such as area, perimeter, circularity, etc.) and gray level first-order statistical features (such as mean, variance, skewness, etc.) of the lateral ventricle are extracted from different views (sagittal, coronal, transverse), and weighted average or multi-statistic (P25, P75, maximum, minimum) fusion is used in combination with multiple views.

[0065] The specific examples of using weighted average or multi-statistic (P25, P75, maximum, minimum) fusion in combination with multiple views are as follows:

[0066] The gray mean value of the sagittal, coronal and transverse planes is extracted, and the gray mean value is shown in Table 1:

[0067] Table 1 Gray mean value of sagittal, coronal and transverse planes

[0068]

[0069] Weighted average fusion is to assign a weight to each view: the weight of the sagittal plane (S) is 0.3, the weight of the coronal plane (C) is 0.4, and the weight of the transverse plane (A) is 0.3, then the fused gray value F is: F = 0.3 x 100 + 0.4 x 120 + 0.3 x 110 = 30 + 48 + 33 = 111. Multi-statistic fusion is to take the three gray mean values of the sagittal plane S, the coronal plane C and the transverse plane A as samples, and extract the following statistical features: ① maximum value Max = 120; ② minimum value Min = 100; ③ quartile P25 ≈ 105, P75 ≈ 115 (here simplified as median estimation), and the final formed feature vector may be: [Max = 120, Min = 100, P25 = 105, P75 = 115].

[0070] Further, the extracted three-dimensional features and multi-view radiomics features are fused to form a high-dimensional feature vector. Then, the mutual information algorithm is used to select the features, identify the most relevant feature subset for the lateral ventricle widening grade, reduce the model input dimension, and retain the key information.

[0071] The example of generating a feature subset is as follows:

[0072] The following 10-dimensional original feature vectors were extracted from the segmentation results. The mutual information algorithm was used to quantify the correlation between each feature and the label. The calculation results are shown in Table 2.

[0073] Table 2 shows the calculation results of using the mutual information algorithm to quantify the correlation between each feature and the label.

[0074]

[0075] A threshold can be set (e.g., greater than 0.1) to ultimately select the feature subsets: F1, F3, F4, F5, F7, F10. This is the feature subset selection process in dimensionality reduction.

[0076] 3.2. Model Prediction

[0077] A multi-class prediction model for lateral ventricle enlargement was constructed using a variety of classic and advanced machine learning methods. First, structural feature vectors filtered by mutual information were input into a classification network. In the implementation, multiple classification algorithms, including Support Vector Machine (SVM), Random Forest, Logistic Regression, K-Nearest Neighbors (KNN), and Multilayer Perceptron (MLP), were used for performance comparison and model ensemble. During model training, 5-fold cross-validation was used to evaluate generalization performance, and grid search was used to systematically optimize the hyperparameters of each model. For the loss function, multi-class cross-entropy was used as the primary optimization objective function to minimize the difference between the predicted class probability distribution and the true label. Furthermore, in the model selection process, multiple indicators such as accuracy, AUC, recall, and F1 score were comprehensively considered, and the model structure and configuration that performed best on the validation set were ultimately selected for deployment in the integrated system. AUC, in medicine, usually refers to the "Area Under Curve," which is mainly used to evaluate diagnostic accuracy, model performance, and the merits of different detection methods. A value closer to 1 indicates higher diagnostic accuracy and better model performance.

[0078] The advantage of the widening prediction model based on three-dimensional features is that it extracts the morphological and texture features of the lateral ventricle and its surrounding white matter region based on the three-dimensional segmentation results, and constructs a high-dimensional radiomics feature expression by combining multi-view statistics. The model can accurately predict the three types of ventricular states: normal, mild widening, and moderate / severe widening.

[0079] Example 2

[0080] Based on the lateral ventricle enlargement prediction method based on the three-dimensional nnUNet network in Example 1, in order to establish alignment relationships between feature maps of different modalities, a cross-modal feature alignment method is further proposed. By comparing loss functions, the feature distributions of different sequences in the same anatomical region are made closer.

[0081] The cross-modal feature alignment method is implemented in the encoder of the nnUNet architecture. The modal feature maps in the encoder of the nnUNet architecture are mapped by convolution dimension reduction and uniformly enter the shared projection space to realize cross-modal feature alignment. The cross-modal feature alignment method is usually implemented in a cross-modal feature alignment module (CM-Align module). The CM-Align module narrows the feature distribution of different sequences in the same anatomical region through a contrast loss. For feature vectors extracted in different modalities for the same anatomical region, the contrast loss between the feature vectors is calculated, and the network parameters are optimized through back propagation to make the feature distributions of different modalities more similar, thereby realizing effective fusion of cross-modal features. The CM-Align module processing process adopts a three-dimensional full-resolution image segmentation network architecture, as shown in Figure 3 .

[0082] Through the contrast loss function, the neural network is optimized so that the feature vectors extracted in different modalities T1 and T2 for the same anatomical region have more similar distributions in the high-dimensional embedding space, that is, the Euclidean distance is smaller. By minimizing the distribution distance of different modal features, the feature representation learned by the network has consistency between different modalities, realizing cross-modal consistency and promoting effective fusion.

[0083] The reason for narrowing the feature distribution of different sequences in the same anatomical region through a contrast loss is explained as follows: different sequences T1, T2, etc. of MRI have different tissue imaging mechanisms, resulting in differences in image feature expression of the same structure in different sequences. If different modal features are directly spliced or fused, the modal difference may cause the feature space to be mixed, affecting the understanding of the nature of the structure and the fusion efficiency of the model. The CM-Align module is used to maximize the similarity of different modal features for the same region, which can suppress irrelevant information specific to the modal and highlight the commonality of the structure itself, thereby improving the performance of subsequent fusion, classification or segmentation tasks.

[0084] Specifically, the CM-Align module is embedded between the two convolutional layers adjacent to stage3 in the encoder backbone of the nnUnet network structure, and preferably embedded between stage3 (the 3rd convolutional layer) and stage4 (the 4th convolutional layer). Each modal feature map is mapped by 1x1x1 convolution dimension reduction and uniformly enters the shared projection space. The contrast loss is used to optimize the feature consistency of the same anatomical structure between modalities and encourage T1 sequence images or T2 sequence images to output similar embedding vectors at the same structure position (if the sagittal T2 and the transverse T1 have significant activation in the left ventricle region, the CM-Align module forces the alignment of their feature responses to form a modal invariant representation).

[0085] The advantages of cross-modal alignment and multi-view fusion are that a cross-modal feature alignment module (CM-Align) is proposed to realize spatial alignment between T1 / T2 and other multi-modal MRIs, integrate multi-directional information of sagittal, coronal and transverse planes, break through the limitation of multi-view but single sequence, and enhance the expression ability of the model to complex structural features.

[0086] Embodiment 3

[0087] The existing prediction of fetal lateral ventricle widening ignores the dynamic characteristics of the fetus in the rapid development process in the second trimester (18-28 weeks), and a static model cannot provide reliable prediction support. Therefore, the development of the fetal lateral ventricle has obvious time characteristics, especially in the second trimester (18-28 weeks), which has a fast growth rate and large structural change. On the basis of the prediction methods in Embodiments 1 and 2, a time sequence dynamic modeling optimization process is further proposed: modeling the evolution trend of the ventricular structure in different time periods and outputting a dynamic change atlas; inputting the time sequence indicators and three-dimensional segmentation results in the dynamic change atlas into the prediction model to obtain the classification results of the optimized lateral ventricle widening degree.

[0088] The time sequence dynamic modeling optimization step is integrated into a time sequence dynamic modeling module, which is fused with the main prediction network to obtain an optimized fusion prediction model. The output result of the model can reflect the changes during the pregnancy. The main goal of the time sequence dynamic modeling module is to model the structural change trend using existing or simulated pregnancy scan sequences, that is, through the time sequence modeling module, the morphological evolution trend of the structure during the pregnancy is "numerically modeled and embedded", which is then used as the input of the prediction model to supplement the limitations of the static prediction model and improve the judgment of the system on time-related indicators such as "expansion speed" and "abnormal growth direction". The overall design consists of three parts: gestational age adaptive grouping, time sequence modeling structure, and dynamic change atlas generation. The time sequence dynamic modeling optimization step is as shown in Figure 5 .

[0089] 1. Gestational age adaptive grouping: The training samples are grouped according to a time interval of 2 weeks, such as GW18-20, GW20-22, etc. According to the development characteristics of the fetal lateral ventricle at different gestational ages, the data is grouped for processing to construct a time period sub-model or set a time embedding, allowing the model to focus on the development of the ventricle in this gestational age period to improve the adaptability of the model to data at different gestational ages.

[0090] 2. Time sequence modeling structure: Construct a continuous gestational age sequence of lateral ventricle three-dimensional features , representing the volume , surface area V , and curvature S of the lateral ventricle at Cdynamic feature embedding of each gestational week , which better captures the variation of the lateral ventricle in the time dimension and the structural changes before and after the time. The dynamic features of each gestational week are represented by formula (2):

[0091] (2)

[0092] where X represents the dynamic features, t represents the gestational week, H t represents the dynamic feature encoding sequence.

[0093] 3. Dynamic change atlas generation: the predicted structure growth rate is output by the LSTM, such as , refers to the change amount of the lateral ventricle volume between two consecutive gestational weeks, is the interval between two scans. And further extrapolate future structural development trends, such as lateral ventricle expansion rate, morphological evolution trajectory, etc. The model automatically associates the baseline features of GW20, quantifies the volume change rate within two weeks. Based on the bidirectional LSTM, the dynamic change atlas of the second trimester is constructed, realizing the personalized tracking of the lateral ventricle development trajectory, providing dynamic features for the prediction model in time sequence. The results of time sequence modeling are input into the widening prediction model in the form of features, and the static structure features are jointly input, and the prediction results optimized by dynamic modeling are output. Among them, the mean square error is used to construct the auxiliary loss function, and the auxiliary loss function is represented by formula (3):

[0094] (3)

[0095] where, is the loss function after adding the time sequence dynamic modeling module. is the volume change predicted by the model, is the volume change calculated according to the label.

[0096] The time sequence indicators in the dynamic change atlas generated by the time sequence dynamic modeling optimization process are used as auxiliary features, which are input into the prediction model, combined with three-dimensional features and multi-view radiomics features, to realize the prediction of fetal lateral ventricle widening state. Through time sequence dynamic modeling analysis of the dynamic change rule of the lateral ventricle, the prediction accuracy is improved, so that the prediction model can more accurately capture the dynamic evolution process of the lateral ventricle widening, thereby improving the accuracy and reliability of the prediction.

[0097] This embodiment mainly embodies the time sequence dynamic modeling mechanism. In view of the rapid development characteristics of the fetal brain in the second trimester, a BiLSTM network is introduced to model the evolution trend of the ventricle structure at different gestational weeks, and an individual dynamic change atlas is output as an auxiliary feature to be fused into a classifier, so as to improve the sensitivity and prediction accuracy of the model to the developmental trend abnormality. The model has high automation, interpretability and good clinical adaptability, and has practical transformation application value.

[0098] Further, the schemes of embodiments 1, 2 and 3 can be combined together, and the prediction model is combined with a multi-model classifier through structured feature extraction and time sequence modeling enhancement, so as to realize accurate prediction of the fetal lateral ventricle widening grade at different gestational weeks.

[0099] Embodiment 4

[0100] On the basis of the methods of the foregoing embodiments 1-3, a method for predicting prognosis risk is further proposed. The method is proposed to further predict the risk of the possible neurodevelopmental outcome of the fetus after birth on the basis of the lateral ventricle widening classification, and to assist doctors in making more reasonable intervention and management strategies before birth. Analysis of the statistical data related to adverse prognosis shows that even isolated mild ventricle widening also has a probability of 7-10% of neurodevelopmental abnormalities after birth; for mild to moderate ventricle widening, the probability of combining other abnormalities is close to 50%; the rate of combined abnormalities of severe widening (>15mm) is as high as 58-65%, and is closely related to severe structural abnormalities such as agenesis of the corpus callosum, Dandy-Walker syndrome, aqueduct stenosis and spina bifida. Accurate prognosis prediction is particularly important for optimizing the timing of intervention and guiding postnatal follow-up.

[0101] On the basis of the widening classification and segmentation results, a comprehensive feature construction strategy based on structure + time features is further introduced, and three-dimensional structure features, development trend indicators and classification results are output to a pre-constructed prognosis prediction model to output the risk contribution ranking of each key prediction factor. The prognosis prediction model includes an interpretive evaluation part.

[0102] Further, the development trend indicators include development speed features, radiomics features and auxiliary labels from historical data. The development speed features include ventricle volume change rate, surface area change rate, fluctuation amplitude of asymmetry over time, etc.; the radiomics features include texture complexity, shape deviation degree, left-right side difference index, periventricular gray matter distribution, etc.; the auxiliary labels from historical data include gestational week correction term, widening classification result, structure change curve embedding, etc. By introducing the development trend indicators, the slight development deviation in the evolution process from mild widening to moderate and severe widening can be effectively captured.

[0103] The prognosis prediction model adopts an integrated classifier strategy, including multiple models such as LightGBM, XGBoost, and support vector machine in parallel, and generates a final prognosis classification grade (good, medium, and poor) by using a weighted fusion method. The prognosis prediction model training is based on real postnatal follow-up data, such as neurodevelopmental scale scores (such as Bayley or Gesell scale), for label fitting, which improves the recognition ability of mild risk and occult structural abnormalities. If there is no follow-up score label, an auxiliary scoring system is constructed based on the co-occurrence of structural abnormalities using a graph model.

[0104] The advantage of intelligent prediction of postnatal prognosis risk is that an individual-level prognosis prediction model is constructed based on three-dimensional structural features, development trend indicators, and classification results, combined with multi-model integration and explanatory evaluation, to output prognosis grades such as good, medium, and poor, realize intelligent prediction of the whole process from image structure to functional risk, and significantly enhance the clinical landing ability of the present application.

[0105] Embodiment 5

[0106] The lateral ventricle widening prediction device based on the three-dimensional nnUNet network, a device structure diagram is shown in Figure 6 The device structure diagram is shown in

[0107] The acquisition module is configured to acquire a magnetic resonance image of the brain.

[0108] The segmentation module is configured to perform three-dimensional segmentation processing on the magnetic resonance image by using a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result. The three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model.

[0109] The prediction module is configured to input the three-dimensional segmentation result into a prediction model to obtain a classification result of lateral ventricle adverse prognosis.

[0110] Further, the segmentation module includes a cross-modal feature alignment unit CM-Align module, which is embedded between stage 3 and stage 4 of the nnUnet encoder backbone. The CM-Align module narrows the feature distribution of different sequences in the same anatomical region through a contrast loss. Specifically, for the feature vectors extracted in different modalities for the same anatomical region, the contrast loss between them is calculated, and the network parameters are optimized through back propagation to make the feature distributions of different modalities more similar, thereby effectively fusing the cross-modal features.

[0111] Preferably, the prediction device further comprises a time sequence dynamic modeling module. The development of the fetal lateral ventricle has obvious time characteristics, especially in the rapid growth and large structural change in the second trimester (18-28 weeks). A time sequence dynamic modeling module that adapts to the changes during pregnancy is specially constructed and integrated with the main prediction network. The gestational age development trend information from the dynamic modeling module is used as an additional input feature to jointly predict with the static structural features. By introducing the time dimension change trend, the sensitivity to the dynamic evolution process of the ventricle is improved, so that the prediction model not only reflects the structure state, but also captures the structure development speed and morphological evolution trajectory.

[0112] Further, a cross-modal feature alignment module is also included, which is embedded in the segmentation module to realize the spatial alignment of features of multiple modalities.

[0113] Preferably, a prognosis risk prediction module is also included. The prognosis risk prediction module integrates SHAP and other explainability techniques to output the risk contribution ranking of each key prediction factor, and supports the generation of prognosis probability distribution, structural change trajectory graph and individualized prognosis evaluation report, which facilitates doctors to make judgments and tracking suggestions on the results. Overall, this module completes the whole process closed loop from anatomical quantification to functional prognosis, and improves the clinical practical value and convertibility of the present application.

[0114] The method and device of the present application combine three-dimensional structure modeling, cross-modal alignment and gestational age dynamic learning, and are significantly superior to traditional two-dimensional segmentation and static judgment methods in early identification of lateral ventricle widening. They show higher sensitivity and discriminability in the 18-22 week gestational period for mild and potential widening cases, and can provide more forward-looking diagnostic basis for the clinic. Their high-precision segmentation capability, structural continuity optimization and development trend prediction show the leading potential in the field of AI-assisted diagnosis of prenatal images. At the same time, the present application proposes a prognosis risk prediction module. As shown in the experimental results of risk prediction, Figure 7 the module further evaluates the possibility of neurodevelopmental abnormalities on the basis of structural typing, identifies fetal individuals with a high risk of structural abnormalities or developmental delay after birth, and provides decision-making basis for early precise intervention, postnatal management and family counseling, which has good medical translation and clinical practice value.

[0115] The beneficial effects of the present application include:

[0116] Three-dimensional segmentation and multi-sequence integration: The invention uses the nnUNet network for three-dimensional automatic segmentation of fetal lateral ventricles, combined with TopoLoss to improve the continuity and accuracy of three-dimensional structure, solving the problem that traditional two-dimensional segmentation cannot capture the overall anatomical morphology of the lateral ventricle. Cross-modal alignment and multi-view fusion: A cross-modal feature alignment module (CM-Align) is proposed to achieve spatial alignment between T1 / T2 and other multi-modal MRI, integrate multi-directional information such as sagittal, coronal and transverse, break through the limitations of multi-view but single sequence, and enhance the model's ability to express complex structural features. Three-dimensional feature-driven widening degree prediction model: Based on three-dimensional segmentation results, morphological and texture features of the lateral ventricle and its surrounding white matter region are extracted, combined with multi-angle statistics, high-dimensional radiomics feature expression is constructed, and through a classification model, the normal, mild widening and moderate / severe widening of the ventricle are accurately predicted. Temporal dynamic modeling mechanism: In view of the rapid development characteristics of the fetal brain in the second trimester, the BiLSTM network is introduced to model the evolution trend of the ventricular structure at different gestational weeks, output individual dynamic change atlas, as an auxiliary feature into the classifier, improve the sensitivity and prediction accuracy of the model to developmental trend abnormalities, with high automation, interpretability and good clinical adaptability, and has practical application value. Intelligent prediction of postnatal prognosis risk: Based on three-dimensional structural features, development trend indicators and classification results, an individual-level prognosis prediction model is constructed, combined with multi-model integration and interpretive evaluation, output good, medium, poor, etc. Prognosis level, realize the whole process of intelligent prediction from image structure to functional risk, significantly enhance the clinical landing ability of the invention. The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of predicting lateral ventricle enlargement, characterized by, The method comprises the following steps: performing three-dimensional segmentation processing on the magnetic resonance image through a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result; the three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model; inputting the three-dimensional segmentation result into a prediction model to obtain a classification result of adverse prognosis of the lateral ventricle; in the three-dimensional segmentation processing, cross-modality feature alignment is adopted: after multiple modality features are subjected to convolution dimension reduction, the multiple modality features are mapped to the same projection space, so that the multiple modality features are aligned in space; the training of the three-dimensional multi-sequence segmentation model by using the topological loss function specifically comprises: extracting a three-dimensional normal vector field from the boundary of the three-dimensional segmentation result output by the three-dimensional multi-sequence segmentation model, and processing the abrupt change of the included angle of adjacent normal vectors in the three-dimensional normal vector field to realize smooth transition of the structure of the prediction result.

2. The method of predicting lateral ventricle enlargement according to claim 1, wherein the cross-modality feature alignment is realized in an encoder of the three-dimensional nnUNet network, and after multiple modality features in the encoder are subjected to convolution dimension reduction, the multiple modality features are mapped to the same projection space, so that the multiple modality features are aligned in space.

3. The method of predicting lateral ventricle enlargement according to claim 2, wherein the encoder realizes convolution dimension reduction through a plurality of convolution modules, and each convolution module adopts the steps of convolution processing, normalization processing, ReLU activation and down-sampling.

4. The method of predicting lateral ventricle enlargement according to claim 2, wherein the cross-modality feature alignment is realized between two adjacent convolution modules in the encoder.

5. The method of predicting lateral ventricle enlargement according to claim 1, wherein the cross-modality feature alignment further comprises: using a contrast loss to optimize the consistency of the features of the same anatomical structure between multiple modalities.

6. The method of predicting lateral ventricle enlargement according to any one of claims 1 to 5, wherein the steps further comprise: resampling the magnetic resonance image in a plurality of directions at the same resolution to obtain resampled data, and inputting the resampled data into the three-dimensional nnUNet network for three-dimensional segmentation processing.

7. A method of predicting lateral ventricle enlargement according to any one of claims 1-5, wherein the steps further comprise a step of time sequence dynamic modeling optimization: obtaining a dynamic change atlas according to the evolution trend of the ventricle structure in different time periods; inputting the time sequence index in the dynamic change atlas and the three-dimensional segmentation result into the prediction model to obtain a classification result of adverse prognosis of the lateral ventricle based on dynamic time sequence.

8. The method of predicting lateral ventricle enlargement according to any one of claims 1 to 5, wherein the steps further comprise a prognosis risk prediction step, specifically comprising: inputting the classification result of the lateral ventricle widening degree into a prognosis risk prediction model to output a prognosis classification level; the prognosis risk prediction model comprises at least two classifiers, the classifiers are connected in parallel, and the results output by each classifier are weighted and summed and then output.

9. A lateral ventricle widening prediction device characterized by comprising: comprise: a collection module configured to collect a magnetic resonance image of a brain; a segmentation module configured to perform three-dimensional segmentation processing on the magnetic resonance image through a three-dimensional multi-sequence segmentation model to obtain a three-dimensional segmentation result; the three-dimensional multi-sequence segmentation model is constructed based on a three-dimensional nnUNet network, and a topological loss function is used to train the three-dimensional multi-sequence segmentation model; a prediction module configured to input the three-dimensional segmentation result into a prediction model to obtain a classification result of adverse prognosis of the lateral ventricle; in the three-dimensional segmentation processing, cross-modality feature alignment is adopted: after multiple modality features are subjected to convolution dimension reduction, the multiple modality features are mapped to the same projection space, so that the multiple modality features are aligned in space; The training of the three-dimensional multi-sequence segmentation model by using the topological loss function specifically comprises: extracting a three-dimensional normal vector field from a boundary of a three-dimensional segmentation result output by the three-dimensional multi-sequence segmentation model, processing an adjacent normal vector angle mutation in the three-dimensional normal vector field, and realizing a smooth transition of a prediction result structure.

10. The lateral ventricle widening prediction device of claim 9, wherein, Further comprising a cross-modal feature alignment module embedded in the segmentation module, used to realize the alignment of features of multiple modalities in space.

11. The lateral ventricle widening prediction device of claim 9, wherein, Further comprising a time sequence dynamic modeling module, which outputs a dynamic change atlas according to the evolution trend of the ventricle structure at different gestational weeks; a time sequence index in the dynamic change atlas is used to input the prediction model together with the three-dimensional segmentation result, to obtain a classification result of the lateral ventricle adverse prognosis based on dynamic time sequence.

12. The lateral ventricle widening prediction device of claim 9, wherein, Further comprising a prognosis risk prediction module, which is used to input the classification result of the lateral ventricle widening degree into a prognosis risk prediction model to output a prognosis classification level; the prognosis risk prediction model comprises at least two classifiers, the classifiers are connected in parallel, and the results output by each classifier are weighted and summed to output.

13. A program product for lateral ventricle widening prediction, characterized in that The program product realizes the lateral ventricle widening prediction method based on the three-dimensional nnUNet network according to any one of claims 1-8 when executed on a computer.

14. A medium characterized by, The storage medium has instructions executable by a processor, and the instructions, when executed by the processor, cause the processor to perform the lateral ventricle widening prediction method according to any one of claims 1-8.

Images