Image auxiliary data processing method and system suitable for neurology department

Abstract

The invention relates to an image auxiliary data processing method and system suitable for the neurology department. The method comprises the following steps: acquiring multi-temporal neurology image data and clinical data to generate a standardized spatio-temporal data set; establishing an incidence matrix of lesion areas and symptoms by extracting dynamic features of the standardized spatio-temporal data set; inputting the matrix into a prediction model to obtain a lesion diffusion direction and symptom risk prediction result, visually displaying prediction information through a dynamic indication diagram, and performing closed-loop verification and model optimization in combination with clinical actual data to realize dynamic tracking of a neurological lesion evolution process and intelligent prediction of a symptom development trend. The continuity of illness state analysis and the reliability of a prediction result are remarkably improved, sequential decision support is provided for clinical diagnosis and treatment, and meanwhile the interpretability of a pathological mechanism is enhanced through visual interaction.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and particularly relates to a method and system for assisting data processing of neurology images. Background Art

[0002] In recent years, significant progress has been made in the auxiliary analysis technology of neurology medical images. Existing methods mainly locate the lesion area through means such as image segmentation and three-dimensional reconstruction, and statically associate the lesion location with typical symptoms by combining a preset rule base. These technologies can help doctors quickly identify abnormal areas and provide basic symptom explanations for patients, but there are still obvious limitations in practical applications.

[0003] On the one hand, traditional methods mostly process image data at a single time point and cannot track the dynamic change process of lesions over time, making it difficult to provide a basis for doctors to predict the development trend of the condition.

[0004] On the other hand, existing visualization schemes focus on the annotation display of the lesion location and lack the dynamic simulation of the lesion diffusion path, the influence range of the nerve function network, and the symptom association strength, resulting in the difficulty for both doctors and patients to intuitively understand the internal relationship between the disease evolution and symptom development.

[0005] In addition, most rely on manual experience to preset lesion-symptom matching rules, and have insufficient flexibility and adaptability in the face of complex and changeable clinical scenarios. Especially when dealing with complex situations such as coexistence of multiple lesions and the intervention of nerve compensation mechanisms, it is difficult to generate individualized prediction and intervention suggestions.

[0006] Therefore, there is an urgent need for an auxiliary analysis method that can integrate temporal and spatial image features, dynamically simulate the law of lesion propagation, and intelligently predict symptom risks to fill the gaps in the current technology in terms of time dimension analysis and clinical decision support. Summary of the Invention

[0007] Based on this, it is necessary to provide a method and system for assisting data processing of neurology images in view of the above technical problems.

[0008] In the first aspect, the present application provides a method for assisting data processing of neurology images, including: S1: Generate a standardized spatio-temporal data set according to the multi-temporal neurology image data and clinical data of the patient; S2: Extract the dynamic features of the lesion area based on the standardized spatio-temporal data set; construct a spatio-temporal symptom association matrix based on the dynamic features; S3: Based on the spatio-temporal symptom association matrix, use the lesion propagation path prediction model to predict the lesion propagation path of the lesion area, and obtain the lesion diffusion direction and symptom risk prediction result; S4: Generate a dynamic lesion indication map based on the lesion spread direction and symptom risk prediction results; S5: By comparing the symptom risk prediction results with clinical data, the lesion transmission path prediction model is iteratively optimized.

[0009] In a second aspect, the present application also provides an image-assisted data processing system applicable to neurology, comprising: A data processing module, used for generating a standardized spatiotemporal data set based on the multi-phase neurological image data and clinical data of the patient; The feature analysis module is used to extract the dynamic features of the lesion area based on the standardized spatiotemporal data set; based on the dynamic features, the spatiotemporal symptom association matrix is ​​constructed; The prediction analysis module is used to predict the lesion propagation path of the lesion area based on the spatiotemporal symptom association matrix and the lesion propagation path prediction model to obtain the lesion diffusion direction and symptom risk prediction results; A visualization module is used to generate dynamic lesion indication maps based on the lesion spread direction and symptom risk prediction results; The optimization module is used to iteratively optimize the lesion propagation path prediction model by comparing the symptom risk prediction results with the clinical data.

[0010] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements a method for image-assisted data processing applicable to neurology as described in the first aspect.

[0011] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for image-assisted data processing applicable to neurology as described in the first aspect is implemented.

[0012] The above-mentioned image-assisted data processing method and system suitable for neurology generates a standardized spatiotemporal data set by collecting multi-phase neurology image data and clinical data; establishes a correlation matrix between lesion areas and symptoms by extracting the dynamic features of the standardized spatiotemporal data set; inputs the matrix into the prediction model to obtain the lesion diffusion direction and symptom risk prediction results, and intuitively displays the prediction information through dynamic indicator diagrams, while combining closed-loop verification and model optimization with actual clinical data, to achieve dynamic tracking of the evolution process of neurological lesions and intelligent prediction of symptom development trends, significantly improve the continuity of disease analysis and the reliability of prediction results, provide time-series decision support for clinical diagnosis and treatment, and enhance the interpretability of pathological mechanisms through visual interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A flowchart of a method for image-assisted data processing applicable to neurology provided by the present invention; Figure 2 A schematic diagram of a process for constructing a spatiotemporal symptom association matrix in an optional embodiment of the present invention; Figure 3 A schematic diagram of a process for extracting dynamic features of a lesion area in an optional embodiment of the present invention; Figure 4 A schematic diagram of a flow chart of predicting the direction of lesion spread and symptom risk prediction results in an optional embodiment of the present invention; Figure 5 A schematic diagram of a process for generating a dynamic lesion indication map in an optional embodiment of the present invention; Figure 6 A schematic diagram of a process for updating a dynamic lesion indication map in an optional embodiment of the present invention; Figure 7 A structural schematic diagram of a neurology image-assisted data processing system provided by the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0016] refer to Figure 1 , which shows a flow chart of a method for image-assisted data processing applicable to neurology provided by the present application, the method comprising the following steps: S1: Generate a standardized spatiotemporal dataset based on the patient's multi-phase neurological image data and clinical data.

[0017] Specifically, neurological images of patients at different time points are obtained from the hospital's imaging department, such as MRI (magnetic resonance imaging), CT (computed tomography), etc. The above image data can be stored in DICOM format, including the patient's head three-dimensional image information. For example, the patient may undergo MRI scans at the time of initial diagnosis (time point T1), mid-treatment (time point T2), and late-treatment (time point T3), and each scan generates a series of image slices. A preliminary quality check of the collected image data can eliminate images with obvious artifacts or damage.

[0018] The clinical data of patients obtained from the hospital's electronic medical record system can include medical history, symptom description, laboratory test results, treatment process records, etc. For example, the patient's medical history may include basic diseases such as hypertension and diabetes, the symptom description may include headache, limb numbness, etc., and the laboratory test results include blood indicators, etc. The clinical data can be structured and converted into a format suitable for computational analysis. For example, the symptom description is converted into a coded form, such as headache coded as "SYM_001", limb numbness coded as "SYM_002", etc., which is convenient for association with image data.

[0019] Multi-phase neurology image data and clinical data are fused. First, the image data is spatially standardized to unify images collected at different time points and different devices into a standard coordinate system. This can be achieved through image registration algorithms, such as mutual information-based registration methods, to align images at different time points in space. Then, the image data is time-standardized. Considering that the time intervals between different visits of patients may be inconsistent, a time interpolation method is used to generate image sequences with equal time intervals. For example, if the original time points are T1, T2, and T3, and the intervals are not equal, an image sequence with one time point per month can be generated by interpolation. Finally, the clinical data is standardized to unify the data units and formats. For example, blood pressure values ​​are unified into millimeters of mercury (mmHg) units, and the frequency of symptom occurrence is unified into standard descriptions such as "daily" and "weekly".

[0020] To build a storage architecture for standardized spatiotemporal datasets, a hierarchical folder structure can be used to classify and store data according to patient ID, time point, and data type (image / clinical). For example, if the patient ID is "PAT_001", the time point is "TIME_001", and the data type is "IMAGE" or "CLINICAL", the corresponding file storage paths are "PAT_001 / TIME_001 / IMAGE / " and "PAT_001 / TIME_001 / CLINICAL / ". Establish data indexes in the database to facilitate quick query and retrieval. Index information can include basic patient information, time point, data type, data processing status, etc. For example, SQL statements can quickly query all patient data with MRI images during treatment.

[0021] S2: Based on the standardized spatiotemporal dataset, the dynamic features of the lesion area are extracted; based on the dynamic features, the spatiotemporal symptom association matrix is ​​constructed.

[0022] Specifically, the image data in the standardized spatiotemporal data set is further preprocessed to improve the accuracy of lesion area detection. The preprocessing steps may include image enhancement, noise removal, etc. For example, the histogram equalization method is used to enhance the contrast of the image to make the lesion area more obvious; the median filter is used to remove Gaussian noise in the image. The image is standardized to normalize the grayscale value of the image to a fixed range, such as [0,1], so that the subsequent algorithm can be uniformly processed; this can be achieved through linear transformation or nonlinear transformation, such as mapping the minimum grayscale value of the image to 0 and the maximum grayscale value to 1.

[0023] Image segmentation algorithms are used to detect lesion areas. For brain neurology images, segmentation algorithms can include threshold-based methods, region growing algorithms, level set methods, etc. For example, threshold-based methods can set a suitable threshold based on the difference in grayscale values ​​between lesion areas and normal tissues to segment images into lesion and non-lesion areas. Combined with deep learning techniques, such as U-Net neural networks, lesion areas are automatically segmented. U-Net is a convolutional neural network designed specifically for medical image segmentation that can learn feature representations in images and accurately segment lesion areas. When training the U-Net model, a large amount of annotated lesion image data is used as a training set, and the network parameters are adjusted through the back propagation algorithm so that the model can adapt to different types of lesions. The segmentation results are post-processed, such as morphological operations (erosion, dilation, etc.), to remove small noise points and irregular areas in the segmentation results, making the boundaries of the lesion areas smoother and more accurate.

[0024] Predefine a variety of dynamic features of the lesion area, including morphological features (such as volume, area, perimeter, etc.), intensity features (such as average gray value, standard deviation, etc.), and texture features (such as grayscale co-occurrence matrix features, grayscale histogram features, etc.). For example, the volume in the morphological feature can be obtained by calculating the number of pixels in the segmented lesion area and combining it with the voxel size of the image; the average gray value in the intensity feature can directly calculate the grayscale mean of the pixels in the lesion area. Then, according to clinical needs and research purposes, select the appropriate feature combination. For studying the growth rate of lesions, you can focus on extracting the changes in morphological features such as volume over time; for studying the changes in tissue characteristics of lesions, you can focus on extracting intensity and texture features.

[0025] Computer vision and image processing algorithms can be used to extract dynamic features of the lesion area. For morphological features, functions in image processing libraries such as OpenCV, such as contour detection, can be used to obtain the boundaries of the lesion area, and then calculate the area, perimeter, etc. For intensity features, statistical calculations are performed directly on the grayscale values ​​of the pixels in the lesion area. For texture features, the grayscale co-occurrence matrix (GLCM) algorithm is used to calculate the texture features of the lesion area, such as contrast, correlation, energy, etc. The GLCM algorithm calculates the grayscale co-occurrence probability of pixel pairs in different directions and distances to obtain a texture feature matrix, and then extracts statistical features from the matrix. The extracted dynamic features are organized according to the time series to form a dynamic feature vector of the lesion area. For example, for the lesion area at each time point, the three features of volume, average grayscale value, and contrast are extracted to form a three-dimensional dynamic feature vector, which forms a feature matrix as the time series changes.

[0026] Establish the association rules between the dynamic features of the lesion area and clinical symptoms. First, based on medical literature and clinical experience, determine which dynamic features may be associated with which symptoms. For example, the rapid growth of the lesion volume may be associated with the aggravation of the patient's headache symptoms, and the change in the grayscale value of the lesion area may be associated with the decline of the patient's cognitive function. Then, define the quantitative method of the association strength, such as using statistical indicators such as correlation coefficient and mutual information to measure the degree of association between dynamic features and symptoms. The correlation coefficient can measure the degree of linear correlation between two variables, and the value range is [-1,1]. The closer the absolute value is to 1, the stronger the correlation; mutual information can measure the degree of nonlinear correlation between two variables, and the larger the value, the stronger the correlation.

[0027] Construct a spatiotemporal symptom association matrix, where the rows of the matrix represent the dynamic features of the lesion area, the columns represent the clinical symptoms, and the matrix elements represent the strength of the association between the two. For example, the element (i, j) in the matrix represents the correlation coefficient between the i-th dynamic feature and the j-th symptom. With the addition of new data and the optimization of the model, the spatiotemporal symptom association matrix can be dynamically updated. When new patient data or new time point data are added, the strength of the association between the dynamic features and the symptoms is recalculated and the matrix elements are updated. At the same time, the association rules and quantification methods can be adjusted according to clinical feedback and research progress to further optimize the construction of the matrix.

[0028] S3: Based on the spatiotemporal symptom association matrix, the lesion propagation path prediction model is used to predict the lesion propagation path of the lesion area to obtain the lesion diffusion direction and symptom risk prediction results.

[0029] Specifically, select a suitable prediction model, such as a physics-based model (such as a finite element analysis model) or a data-driven model (such as a long short-term memory network LSTM, Transformer and other deep learning models). For the prediction of the propagation path of neurological lesions, considering the characteristics of time series data, the LSTM model is a better choice because it can capture long-term dependencies in time series. Collect a large amount of case data with known lesion propagation paths as a training set, which can include multi-phase image data, dynamic feature sequences, clinical symptom changes, etc. of patients. Preprocess and annotate the training data, and the annotation content includes the actual propagation path of the lesion and the symptom changes. Use the training data to train the selected model, and adjust the model's hyperparameters (such as learning rate, number of hidden layer neurons, etc.) and optimization algorithms (such as Adam, RMSprop, etc.) to enable the model to accurately learn the laws of the lesion propagation path. During the training process, use methods such as cross-validation to evaluate the performance of the model to prevent overfitting and underfitting.

[0030] The input of the model can include the dynamic feature sequence in the standardized spatiotemporal data set, the initial lesion area information (such as location, size, etc.), and the patient's clinical data (such as age, gender, underlying diseases, etc.). For example, the dynamic feature sequence can be a time series matrix, each row represents a dynamic feature vector at a time point; the initial lesion area information can be a vector containing the initial position coordinates and size parameters of the lesion; the clinical data can be a vector containing the patient's age, gender code, etc. The output of the model can include the predicted lesion diffusion direction (such as the diffusion direction expressed in vector form) and the symptom risk prediction results (such as the risk probability of different symptoms). For example, the diffusion direction can be a three-dimensional vector, indicating the main diffusion direction of the lesion in space; the symptom risk prediction result can be a probability vector, each element of which corresponds to the probability of occurrence of a symptom.

[0031] Use the trained lesion propagation path prediction model to predict the new standardized spatiotemporal data set. Input the model input data into the model to calculate the lesion spread direction and symptom risk prediction results. Verify and evaluate the prediction results, using appropriate evaluation indicators, such as mean square error (MSE) to evaluate the accuracy of the spread direction prediction, and the area under the receiver operating characteristic curve (ROC) (AUC) to evaluate the accuracy of the symptom risk prediction. The effectiveness of the model can also be verified by comparing it with the actual lesion propagation path and symptom changes.

[0032] Design a visualization scheme for the prediction results so that doctors can intuitively understand the spread trend of the lesion and the symptom risk. For the direction of lesion spread, the spread direction can be represented by arrows on the three-dimensional brain map model, and the thickness of the arrow can represent the intensity of the spread. For example, using visualization software such as Paraview, the predicted diffusion direction is superimposed on the patient's brain MRI image to generate an intuitive three-dimensional visualization. For symptom risk prediction results, the probability of occurrence of different symptoms can be displayed in the form of bar graphs, heat maps, etc. For example, a symptom risk heat map is generated, with the horizontal axis representing different symptom names and the vertical axis representing risk probability. The depth of color represents the probability, which is convenient for doctors to quickly identify high-risk symptoms.

[0033] S4: Generate a dynamic lesion indication map based on the lesion spread direction and symptom risk prediction results.

[0034] Specifically, the multi-layer structure of the dynamic lesion indication map can include a basic image layer (such as a patient's brain anatomical image), a lesion area layer (displaying lesion areas at different periods), a diffusion path layer (displaying predicted lesion diffusion paths), a symptom risk layer (displaying symptom risk areas), etc. For example, the basic image layer can be a brain MRI image of the patient at the latest time point as a background image; the lesion area layer can be a superimposed display of lesion areas at different periods, and lesions at different periods are distinguished by different colors. Lesion data of different time series, predicted diffusion path data, symptom risk data, etc. are fused into the corresponding layers. For example, the predicted diffusion path data is drawn on the diffusion path layer in the form of transparent lines, and the color and thickness of the lines can be adjusted according to the diffusion intensity and probability.

[0035] It is also possible to design interactive functions for dynamic lesion indication maps, so that users (such as doctors) can easily browse and analyze lesion information. For example, a timeline control is provided, so that users can view lesion conditions at different periods by dragging the timeline; a layer control button is provided, so that users can freely choose to display or hide a layer; and operation tools such as zooming and panning are provided, so that users can view the details of the lesion area in detail.

[0036] Select appropriate visualization tools and software libraries to generate dynamic lesion indication maps. For example, use Matplotlib, Mayavi and other libraries in Python for two-dimensional and three-dimensional visualization; use the Three.js library in JavaScript for interactive visualization on the web. Write program code to draw the data according to the designed layer structure and visualization scheme. For example, use Matplotlib's imshow function to draw the base image layer, use the plot function to draw the lines of the diffusion path layer, and use the scatter function to draw the marker points of the symptom risk layer.

[0037] Determine the output format and application scenarios of the dynamic lesion indication map. The output format may include static images (such as PNG and JPEG formats), dynamic videos (such as MP4 formats), interactive web applications, etc. Application scenarios may include clinical diagnosis assistance, surgical planning, patient education, scientific research, etc. For example, the generated dynamic lesion indication map can be embedded in the hospital's clinical diagnosis system for doctors to refer to during the diagnosis process; it can also generate a video file to explain the disease progression to patients.

[0038] S5: By comparing the symptom risk prediction results with clinical data, the lesion transmission path prediction model is iteratively optimized.

[0039] Specifically, collect feedback from doctors on the prediction results and dynamic lesion indication maps. Feedback can include evaluations of prediction accuracy, visualization effects, clinical practicality, etc. For example, doctors can point out whether there is a deviation between the predicted lesion spread direction and the actual observed direction, and whether the dynamic lesion indication map can help them better understand the condition. Collect the patient's actual disease progression data as a basis for verifying the accuracy of the prediction model. For example, the patient's new imaging examination results and symptom change records during the subsequent treatment process can be compared and analyzed with the previous prediction results.

[0040] Organize and analyze the collected feedback data to extract useful information. For example, classify and count the doctors' feedback to understand their main concerns and improvement directions for the model; pre-process the patients' actual condition data to make them comparable with the predicted data. Integrate the feedback data with the original standardized spatiotemporal data set, spatiotemporal symptom association matrix, lesion propagation path prediction model, etc. to form a new training and optimization data foundation. For example, add the patient's new imaging data to the standardized spatiotemporal data set, and use the doctor's feedback as new supervisory information for retraining the model.

[0041] Select appropriate optimization methods based on the characteristics of feedback data and the performance bottleneck of the model. For example, if the model has large errors in predicting certain types of lesions, you can use transfer learning to migrate the parameters of pre-trained models in other related fields to the current model for targeted fine-tuning; if the generalization ability of the model is insufficient, you can use regularization techniques (such as L1 / L2 regularization) to prevent overfitting. Adjust the structure and parameters of the model, such as increasing or decreasing the number of layers and neurons in the model, changing the activation function, etc. For example, for the LSTM model, you can try to increase the number of neurons in the hidden layer to improve the model's ability to fit complex time series; or try to use different activation functions (such as ReLU, Tanh, etc.) to observe the impact on model performance.

[0042] Establish an evaluation index system for model optimization effects, including classification performance indicators such as prediction accuracy, recall rate, F1 score, and efficiency indicators such as computing resource consumption and response time. For example, for lesion propagation path prediction, the Euclidean distance between the predicted result and the actual result can be calculated. The smaller the distance, the more accurate the prediction; for symptom risk prediction, the cross entropy loss between the predicted probability distribution and the actual occurrence can be calculated. Use cross-validation, leave-one-out validation and other methods to comprehensively evaluate the optimized model to ensure that the model can maintain good performance in different data sets and application scenarios. For example, divide the data set into training set, validation set and test set, use the training set to train the model, adjust the model parameters with the validation set, and evaluate the final model performance with the test set; or use k-fold cross-validation to divide the data set into k subsets, train with k-1 subsets each time, test with 1 subset, repeat k times, and take the average performance index as the final evaluation result.

[0043] The above-mentioned image-assisted data processing method suitable for neurology generates a standardized spatiotemporal data set by collecting multi-phase neurology image data and clinical data; extracts the dynamic characteristics of the standardized spatiotemporal data set to establish a correlation matrix between lesion areas and symptoms; inputs the matrix into the prediction model to obtain the lesion diffusion direction and symptom risk prediction results, and intuitively displays the prediction information through dynamic indicator diagrams. At the same time, closed-loop verification and model optimization are carried out in combination with actual clinical data to achieve dynamic tracking of the evolution process of neurological lesions and intelligent prediction of symptom development trends, significantly improve the continuity of disease analysis and the reliability of prediction results, provide time-series decision support for clinical diagnosis and treatment, and enhance the interpretability of pathological mechanisms through visual interaction.

[0044] refer to Figure 2 In an optional embodiment, S2 includes the following steps: S21: Based on the 4D convolutional neural network, the spatiotemporal feature extraction and processing of the standardized spatiotemporal data set is performed to obtain the dynamic features of the lesion area; the dynamic features include volume change features, position coordinate change features, and diffusion velocity change features. Specifically, the input layer of the 4D convolutional neural network is designed to adapt to the dimensions of the standardized spatiotemporal dataset. The input data is a four-dimensional tensor, including three spatial dimensions (such as x, y, z axes) and one time dimension, with a shape of (number of time points, height, width, depth). For example, for spatiotemporal data containing T time points and an image size of H×W×D at each time point, the size of the input layer is (T, H, W, D). The input data is normalized to normalize the pixel values ​​to the range of [0,1] or [-1,1] to accelerate the convergence speed of the network and improve the training stability. The normalization formula is: Normalized pixel value = (original pixel value - minimum pixel value) / (maximum pixel value - minimum pixel value).

[0045] Construct a combination module of multiple convolutional layers and pooling layers to automatically extract spatiotemporal features. The convolutional layer uses a 4D convolution kernel to perform convolution operations in the time and space dimensions. The convolution kernel size (3, 3, 3, 3) means taking 3 consecutive time points in the time dimension and taking a 3×3×3 cubic area in the space dimension for convolution. The pooling layer uses maximum pooling or average pooling operations to reduce the resolution of the feature map, reduce the amount of calculation, and retain important features. For example, in the time dimension, a maximum pooling with a step size of 2 is used to halve the number of time points; in the space dimension, a maximum pooling of 2×2×2 is used to reduce the spatial resolution by half.

[0046] Adding an activation function, such as ReLU (Rectified Linear Unit), after the convolutional layer introduces nonlinearity, enabling the network to learn complex feature representations. The ReLU function is defined as: ReLU(x) = max(0, x), which sets negative values ​​to zero and keeps positive values ​​unchanged. It has the advantages of simple calculation and alleviating the gradient vanishing problem.

[0047] Perform data augmentation operations on the standardized spatiotemporal dataset, such as random rotation, translation, flipping, etc., to expand the training dataset and improve the generalization ability of the model. For example, randomly rotate the image in the spatial dimension (±15 degrees) to generate new training samples so that the model can adapt to lesion characteristics in different directions. Divide the enhanced data into training set, validation set, and test set in a ratio of 7:2:1 to ensure the reliability of the performance evaluation of the model on an independent test set.

[0048] Use the training set to train the 4D convolutional neural network and define a suitable loss function, such as mean square error loss (MSE) for regression tasks and cross entropy loss for classification tasks. Depending on the task objectives, a custom loss function can be used to comprehensively consider the accuracy of dynamic feature extraction and the correlation with clinical data. Select an optimization algorithm, such as the Adam optimizer, and set hyperparameters such as learning rate and batch size. For example, the initial learning rate is set to 0.001, the batch size is 32, all training data are trained once per epoch, and the network weights are optimized through multiple epoch iterations.

[0049] The trained 4D convolutional neural network can automatically extract the dynamic features of the lesion area from the standardized spatiotemporal data. The output layer of the network is designed to match the target dynamic feature dimension. For example, the volume change feature, position coordinate change feature and diffusion speed change feature are extracted. The output layer can contain three neurons, corresponding to the predicted values ​​of these three features. The extracted dynamic features are post-processed, such as smoothing to remove noise fluctuations, to obtain a smooth dynamic feature curve, which more accurately reflects the evolution trend of the lesion. For example, the moving average method is used to smooth the volume change feature, and the window size is set to 3 time points.

[0050] S22: Perform feature fusion processing on the dynamic features and the symptom records in the clinical data to generate the first correlation matrix Specifically, a feature-level fusion method can be used to directly fuse dynamic features and symptom records in clinical data at the feature level. First, the dynamic features and clinical data are normalized so that they have the same numerical range and dimension. The dynamic feature vector and the symptom record vector are concatenated to form a new fused feature vector. For example, the dynamic features include volume change features (V), position coordinate change features (X, Y, Z), and diffusion velocity change features (S), and the symptom records include symptom codes (SYM1, SYM2,…, SYMn). The fused feature vector is [V, X, Y, Z,S, SYM1, SYM2,…, SYMn].

[0051] In the feature fusion process, the importance differences of different features are taken into account, and weight coefficients are introduced to weight each feature. The weight coefficients can be pre-set through expert knowledge or automatically learned through data-driven methods. For example, the ReliefF algorithm is used to evaluate the correlation between each feature and the lesion area and symptoms, and features with high correlation are given larger weights, and vice versa. The ReliefF algorithm evaluates the importance of features by calculating the weight differences of features in samples of different categories.

[0052] By defining the association rules between dynamic features and symptom records, we can determine which dynamic features may be associated with which symptoms based on medical knowledge and clinical experience. For example, the volume change feature may be positively correlated with the symptom "headache", and the diffusion velocity change feature may be negatively correlated with the symptom "limb weakness". To quantify the association relationship, we use statistical indicators such as correlation coefficient and mutual information to measure the degree of association between dynamic features and symptom records. The correlation coefficient calculation formula is: correlation coefficient = covariance (dynamic feature, symptom record) / (dynamic feature standard deviation × symptom record standard deviation), the value range is [-1,1], and the closer the absolute value is to 1, the stronger the association.

[0053] Construct the first association matrix, the rows of which represent the fused features (including dynamic features and symptom records), the columns represent different lesion areas and symptom combinations, and the matrix elements represent the strength of the association between the two. For example, the element (i, j) in the matrix represents the correlation coefficient between the i-th fused feature and the j-th lesion area and symptom combination. Store the first association matrix in the database to facilitate the subsequent multi-head attention mechanism processing and the generation of the spatiotemporal symptom association matrix. When storing, the matrix dimensions, element values, and corresponding features, lesion areas, and symptom information can be recorded to ensure the integrity and traceability of the data.

[0054] S23: The correlation between the lesion area and the symptoms in the first correlation matrix is ​​weighted by the multi-head attention mechanism to obtain a second correlation matrix containing the correlation weights between the lesion area and the symptoms as the spatiotemporal symptom correlation matrix.

[0055] Specifically, the attention mechanism is a neural network module that simulates human visual attention. It can automatically learn the correlation between input features, assign different weights to different features, and make the model pay more attention to important features. In the field of natural language processing, the attention mechanism is widely used in tasks such as machine translation and text generation, and can effectively capture long-distance dependencies between words. In this method, the attention mechanism is applied to the analysis of the correlation between the lesion area and symptoms to improve the model's ability to model complex associations.

[0056] The multi-head attention mechanism consists of multiple parallel attention heads. Each attention head independently calculates the attention weights between the input features, and then concatenates or sums the outputs of multiple attention heads to obtain the final attention output. Each attention head includes three vectors: query, key, and value. The input features are mapped to these three vector spaces through linear transformation. The attention weight calculation formula is: Attention weight = softmax ((query × key^T) / √d_k), where d_k is the dimension of the key vector, and the softmax function normalizes the weight to a probability distribution.

[0057] The first association matrix is ​​used as the input feature of the multi-head attention mechanism, and each element represents the strength of association between a feature (dynamic feature or symptom record) and the combination of lesion area and symptom. The input features are normalized and dimensionally adjusted to meet the input requirements of the multi-head attention mechanism. For example, the shape of the first association matrix is ​​adjusted to (number of samples, number of features, embedding dimension), and the embedding dimension represents the vector representation length of each feature.

[0058] In the multi-head attention mechanism, each attention head independently calculates the attention weights between the input features to capture the correlation between different features. Through the synergy of multiple attention heads, the complex correlation between the lesion area and the symptoms can be modeled more comprehensively. The outputs of multiple attention heads are concatenated or summed to obtain the final attention output as the second correlation matrix containing the correlation weights between the lesion area and the symptoms. The dimension of the second correlation matrix is ​​the same as the first correlation matrix, but the matrix elements represent the correlation weights after redistribution by the multi-head attention mechanism, which can better reflect the true correlation strength between the lesion area and the symptoms.

[0059] The second correlation matrix is ​​post-processed, such as normalization to make the weight value in the range of [0,1], to facilitate subsequent visualization and interpretation. For example, the Min-Max normalization method is used to map the matrix element values ​​to the interval [0,1]. The second correlation matrix is ​​used as a spatiotemporal symptom correlation matrix in subsequent lesion propagation path prediction and symptom risk prediction tasks to provide the model with more accurate lesion and symptom association information and improve the reliability and accuracy of the prediction.

[0060] refer to Figure 3 In an optional embodiment, S21 includes the following steps: S211: Extract lesion structural features by performing 3D convolution processing on the standardized spatiotemporal dataset in the spatial dimension.

[0061] Specifically, the input layer of the 3D convolutional network is designed to match the spatial dimension of the standardized spatiotemporal dataset. The input data is a three-dimensional tensor representing a neurological image at a single time point, with a shape of (height, width, depth). For example, for a brain MRI image, its size may be 256×256×256, and the input layer needs to adapt to such a three-dimensional data structure. The input image is preprocessed, including normalization and data augmentation. Normalization adjusts the pixel values ​​to the range of [0,1], and data augmentation expands the training data through random rotation, translation and other operations to improve the generalization ability of the model.

[0062] Construct multiple layers of 3D convolutional layers, using different sizes and numbers of convolution kernels in each layer. For example, the first layer uses 32 3×3×3 convolution kernels, and the second layer uses 64 3×3×3 convolution kernels. The convolution operation can automatically extract local features in the image. Add activation functions such as ReLU after each convolutional layer to introduce nonlinearity. The ReLU function is defined as: ReLU(x)=max(0, x), which can accelerate network convergence and alleviate the gradient vanishing problem.

[0063] Add a pooling layer, such as a max pooling layer, after the convolutional layer to reduce the resolution of the feature map and reduce the amount of computation. For example, use a 2×2×2 max pooling to halve the spatial resolution. Add a batch normalization layer to normalize each batch of data, stabilize the network training process, and increase the convergence speed.

[0064] After the forward propagation of the 3D convolutional network, a series of 3D feature maps are generated. These feature maps capture the structural information of the lesion area, such as shape, size, edge features, etc. For example, some feature maps may highlight the boundary of the lesion, while others may emphasize the texture inside the lesion. The number of channels of each feature map increases with the depth of the network, and the number of channels represents the diversity of the extracted features. Shallow networks extract low-level features (such as edges and corners), and deep networks extract high-level features (such as the overall structure of the lesion).

[0065] Perform global average pooling or maximum pooling on the three-dimensional feature map to compress each feature map into a one-dimensional vector. For example, for a feature map of size H×W×D, calculate the average or maximum value of each channel to obtain a vector with a length equal to the number of channels. These vectors are combined into a lesion structure feature vector, which fully describes the structural characteristics of the lesion in space and provides feature input of the spatial dimension for subsequent spatiotemporal fusion.

[0066] S212: Modeling the lesion evolution trend by processing the standardized spatiotemporal dataset with an LSTM network in the time dimension.

[0067] Specifically, the input layer of the LSTM network is constructed to adapt to the time dimension of the standardized spatiotemporal dataset. The input data is a two-dimensional sequence with the shape of (number of time points, feature dimension), where the feature dimension can be the length of the lesion structure feature vector extracted by the 3D convolutional network. The time series data is normalized to ensure that the numerical range of the input data is suitable for the training of the LSTM network. For example, the time series data is scaled to the range of [0,1] or [-1,1].

[0068] Stack multiple LSTM layers, each containing a certain number of LSTM units. For example, the first LSTM layer contains 128 units and the second layer contains 64 units. LSTM units control the flow of information through forget gates, input gates, and output gates, and can effectively handle long-term dependencies in time series. Add dropout layers between LSTM layers to prevent overfitting. For example, set the dropout rate to 0.2, randomly discard 20% of neuron connections, and enhance the generalization ability of the model.

[0069] Design the output layer of the LSTM network and determine the output dimension according to the task requirements. For example, if the task is to predict the evolution trend of the lesion, the output layer may contain the same number of neurons as the lesion structure feature dimension, and directly output the predicted feature vector. Add an activation function after the output layer, such as sigmoid or tanh, and choose the appropriate activation function according to the specific task to ensure that the output data conforms to the expected distribution range.

[0070] The lesion structure feature vector at each time point is input into the LSTM network, and the network automatically learns the changing patterns of features in the time series. For example, the LSTM network can capture the growth trend of lesion volume over time, the migration pattern of location, etc. During the training process, the feature data of historical time points are used to predict the features of future time points, and the network weights are optimized by minimizing the prediction error. For example, the mean square error (MSE) is used as the loss function, the difference between the predicted features and the real features is calculated, and the weights are adjusted by backpropagation.

[0071] The trained LSTM network can generate an evolution trend curve of the lesion. For each lesion structural feature, its predicted value in the time dimension is output to form a curve. For example, a curve showing the change of lesion volume over time is generated to intuitively show the growth or shrinkage trend of the lesion. The evolution trend curve is smoothed to remove possible noise fluctuations and make the curve smoother and more continuous. For example, the moving average method or Savitzky-Golay filter is used to smooth the curve to improve the readability and accuracy of the curve.

[0072] S213: Perform spatiotemporal fusion processing on the lesion structure characteristics and lesion evolution trends to generate a dynamic lesion evolution map.

[0073] Specifically, the lesion structure feature vector extracted by the 3D convolutional network and the lesion evolution trend vector generated by the LSTM network are directly concatenated. For example, if the length of the lesion structure feature vector is 128 and the length of the lesion evolution trend vector is 64, the length of the concatenated spatiotemporal feature vector is 192. In the concatenated feature vector, the first part of the elements represents the spatial structure information of the lesion, and the second part of the elements represents the temporal evolution information of the lesion, and the two together constitute a complete spatiotemporal feature description.

[0074] Introduce weight coefficients to weight spatial features and temporal features separately and then fuse them. Weight coefficients can be set by expert knowledge or learned by data-driven methods. For example, based on the importance evaluation of features, different weights are assigned to lesion structure features and evolution trend features to highlight more important feature parts. Use the attention mechanism to automatically learn the weight of each feature in spatiotemporal fusion. For example, use a small neural network model to input spatiotemporal features, output the attention weight of each feature, and perform feature fusion based on the weights.

[0075] Construct a dynamic lesion evolution map to visualize the changes of lesions in time and space. The map can be in the form of a three-dimensional brain map superimposed on a timeline. The brain map at each time point shows the structural characteristics of the lesion, and uses color, lines and other elements to indicate the evolution trend of the lesion. In the map, different shades of color are used to indicate the size changes of the lesion volume, and arrows are used to indicate the migration direction and speed of the lesion location. For example, the darker the color, the larger the lesion volume, and the longer the arrow, the faster the location changes. An interactive visualization interface is designed so that users can observe the dynamic evolution of the lesion in detail by sliding the timeline, rotating the brain map, zooming in and out, etc., which is convenient for doctors to analyze and diagnose the condition.

[0076] The data of the dynamic lesion evolution map is stored in the database, including the lesion structure characteristics, evolution trend information and parameters of the visualization elements at each time point. For example, the 3D brain map data, color mapping table, arrow coordinates and length and other information at each time point are stored. A data index and query mechanism can be established to facilitate the rapid retrieval of the dynamic lesion evolution map of a specific patient. For example, query by patient ID, time range and other conditions, and quickly load the corresponding map data for display and analysis.

[0077] S214: extracting the volume change characteristics, position coordinate change characteristics and diffusion speed change characteristics of the lesion area from the dynamic lesion evolution map.

[0078] Specifically, the volume of the lesion area at each time point is calculated from the dynamic lesion evolution map. The volume calculation is based on the voxel size of the image and the number of pixels in the lesion area. For example, if the voxel size of the image is 1mm×1mm×1mm and the lesion area contains 1000 pixels, the volume is 1000mm³. The volume calculation results are smoothed to remove possible measurement noise. For example, the moving average method is used to smooth the volume data with 3 time points as a window to obtain a smoothed volume change curve.

[0079] Extract volume change features, including absolute volume value, volume change rate, etc. The volume change rate can be calculated by the difference between the volumes at adjacent time points. For example, volume change rate = (volume at current time point - volume at previous time point) / volume at previous time point. Represent the volume change feature as a vector containing the volume values ​​and change rates at multiple time points to form a volume change feature sequence for subsequent analysis and modeling.

[0080] Determine the centroid coordinates of the lesion area as a representative of its position. The centroid coordinate calculation is based on the weighted average of the coordinates of the pixels in the lesion area. For example, for the lesion area in a two-dimensional image, the centroid abscissa = Σ (pixel abscissa × pixel intensity) / Σ pixel intensity, and the same applies to the ordinate; a similar expansion is performed for a three-dimensional image. Smooth the centroid coordinates to reduce the influence of measurement errors and noise. For example, use exponential smoothing to smooth the centroid coordinate sequence to make the position change curve smoother.

[0081] Extract the position coordinate change features, including the absolute value of the centroid coordinates, the coordinate change amount, the migration speed, etc. The coordinate change amount is the difference between the centroid coordinates at adjacent time points, and the migration speed is the coordinate change amount divided by the time interval. The position coordinate change feature is represented as a multidimensional vector, which contains the coordinate change information in different directions (such as x, y, and z axes), forming a position coordinate change feature sequence, which is used to describe the migration pattern of the lesion in space.

[0082] Calculate the diffusion rate of the lesion based on the expansion of the lesion area at different time points. The diffusion rate can be obtained by dividing the maximum expansion distance of the lesion boundary by the time interval. For example, at two adjacent time points, measure the maximum expansion distance of the lesion boundary in a certain direction and divide it by the time interval to obtain the diffusion rate in that direction. Use multi-directional diffusion rate calculation to comprehensively evaluate the overall diffusion trend of the lesion. For example, calculate the diffusion rate of the lesion in the three axes of x, y, and z respectively, and take the average or maximum value as the comprehensive diffusion rate.

[0083] Extract the diffusion velocity change characteristics, including the absolute value of the diffusion velocity, acceleration, etc. The diffusion velocity acceleration is the difference between the diffusion velocities at adjacent time points divided by the time interval. The diffusion velocity change characteristics are represented as a vector, which contains the diffusion velocity and acceleration information at multiple time points, forming a diffusion velocity change characteristic sequence, which is used to analyze the diffusion dynamics characteristics of the lesion.

[0084] refer to Figure 4 In an optional embodiment, S3 includes the following steps: S31: Construct an initial graph structure with the lesion area as nodes and the diffusion path as edges; the attributes of the nodes include the volume, position coordinates and diffusion speed of the lesion area, and the attributes of the edges include the diffusion direction and time interval; the lesion propagation path prediction model includes the initial graph structure and the graph attention network.

[0085] Specifically, the lesion area is defined as a node in the graph structure, and each node represents an independent lesion area. The attributes of the node include the volume, position coordinates and diffusion speed of the lesion area, which are obtained from the dynamic features extracted in step S21. For example, the attributes of node A are volume V_A, position coordinates (X_A, Y_A, Z_A) and diffusion speed S_A. The node attributes are normalized, and the attribute values ​​of different dimensions and magnitudes are mapped to the same numerical range, such as [0,1], to facilitate subsequent graph neural network processing. The normalization formula is: normalized attribute value = (original attribute value - attribute minimum value) / (attribute maximum value - attribute minimum value).

[0086] The diffusion path between lesion areas is defined as an edge in the graph structure, which connects two nodes (lesion areas). The attributes of the edge include the diffusion direction and time interval. The diffusion direction is calculated by the change of the position coordinates between the lesion areas, and the time interval is obtained from the standardized spatiotemporal dataset. For example, the attributes of edge AB are the diffusion direction vector D_AB and the time interval T_AB. The attributes of the edge are encoded and normalized. The diffusion direction vector is normalized to a unit vector, and the time interval is normalized to the range [0,1] to ensure the numerical stability and consistency of the edge attributes.

[0087] Design a graph structure construction algorithm to automatically generate the initial graph structure based on the standardized spatiotemporal dataset and the extracted dynamic features. The algorithm steps may include: 1) Traverse all lesion areas, create corresponding nodes, and assign attribute values.

[0088] 2) For each pair of lesion regions, calculate the diffusion path and attributes between them and create corresponding edges.

[0089] 3) Integrate all nodes and edges into a graph data structure, such as using an adjacency list or adjacency matrix.

[0090] During the construction process, the spatial and temporal relationships between lesion regions are considered to ensure that the graph structure can accurately reflect the propagation pattern of the lesions. For example, edges are only created between lesion regions that are sequential in time and adjacent in space.

[0091] The initial graph structure is stored in a database using a graph database (such as Neo4j) or a special data type in a relational database (such as PostgreSQL's Adjacency List model). The stored content includes node information (such as node ID, attribute value), edge information (such as the edge's starting node ID, end node ID, attribute value), and the overall structure of the graph (such as the number of nodes, the number of edges, etc.). An index and query mechanism for the graph structure is established to facilitate rapid retrieval and manipulation of nodes and edges in the graph. For example, the node attributes and connected edges can be quickly queried by node ID, and a specific set of edges can be filtered by edge attributes (such as the time interval range).

[0092] S32: The association weights in the spatiotemporal symptom association matrix are input into the initial graph structure as node attributes, and the propagation probability between nodes is calculated through the message passing mechanism of the graph attention network to generate an initial lesion propagation graph containing the propagation weights.

[0093] Specifically, the input layer of the graph attention network is designed to receive the node attributes and edge attributes in the initial graph structure as input. The input data is represented as a node feature matrix and an edge feature matrix. The shape of the node feature matrix is ​​(number of nodes, node attribute dimension), and the shape of the edge feature matrix is ​​(number of edges, edge attribute dimension). Adding an embedding layer maps the input node attributes and edge attributes to an embedding space of fixed dimension to facilitate subsequent graph neural network processing. The embedding layer can use linear transformation or nonlinear transformation (such as MLP). For example, the node attribute dimension is mapped from the original 3 (volume, position coordinates, diffusion speed) to an embedding dimension of 64.

[0094] Multiple layers of graph attention layers are stacked, each layer contains multiple attention heads, which are used to calculate the propagation probability between nodes. Each attention head independently calculates the attention weights between nodes, and then the outputs of multiple attention heads are concatenated or summed to obtain the final attention output. In the graph attention layer, each node calculates the attention weight based on the embedded features and edge attributes of its neighbor nodes. The weight represents the degree of influence of the neighbor node on the current node. The attention weight calculation formula is: Attention weight = softmax ((query × key^T) / √d_k), where query and key are vectors generated from the neighbor node embedded features and edge attributes, and d_k is the dimension of the key vector.

[0095] Design the output layer of the graph attention network and output the propagation probability matrix between each node. The shape of the propagation probability matrix is ​​(number of nodes, number of nodes), and the matrix element (i, j) represents the probability of node i propagating to node j. Add normalization processing after the output layer to ensure that the sum of the elements in each row of the propagation probability matrix is ​​1, indicating that the sum of the probabilities of propagating from one node to all other nodes is 1. The normalization formula is: Propagation probability (i, j) = Propagation probability (i, j) / Σ Propagation probability (i, k) (k is all nodes).

[0096] The association weights in the spatiotemporal symptom association matrix are input into the initial graph structure as node attributes and fused with the original node attributes. For example, the association weight values ​​are added to the node attribute vector to form a new node attribute vector, which is then mapped to the embedding space through the embedding layer. The fused node attributes are normalized to ensure that the data input to the graph attention network has a suitable value range and distribution.

[0097] In the graph attention network, each node performs message passing based on the embedded features and edge attributes of its neighbor nodes. The message passing process may include: 1) Each neighbor node generates a message vector, which is calculated by its embedded features and edge attributes through a specific function (such as linear transformation).

[0098] 2) The current node receives the message vectors of all neighboring nodes and calculates the attention weight of each message vector. The weight indicates the importance of the message to the current node.

[0099] 3) Perform weighted summation on the message vectors according to the attention weights to obtain the updated embedding features of the current node.

[0100] The above process is repeated for multiple rounds so that the embedded features of the nodes can capture more extensive contextual information and propagation relationships in the graph structure.

[0101] After multiple rounds of message passing and attention calculation, the graph attention network outputs a propagation probability matrix. This matrix represents the probability of each node (lesion area) in the graph propagating to other nodes, reflecting the propagation trend and possibility of lesions between different areas. The propagation probability matrix is ​​post-processed, such as removing the minimum value (the probability below a certain threshold is considered to be 0), simplifying the matrix representation, and highlighting the main propagation paths and probabilities.

[0102] S33: performing constraint correction processing on the lesion propagation graph based on the lesion propagation rules predefined in the medical knowledge base to obtain a corrected lesion propagation graph.

[0103] Specifically, we collect multi-source data such as medical literature, clinical guidelines, and expert experience to build a medical knowledge base. The knowledge base includes common propagation patterns of lesions, the propagation patterns of specific lesion types in different locations, and the typical associations between lesions and symptoms. We organize the collected knowledge in a structured manner to form a rule base. For example, a rule can be expressed as "If the lesion type is X and appears at location Y, it usually propagates to location Z with a probability of P", or "When the lesion volume exceeds V, the diffusion rate will increase significantly."

[0104] The rules in the medical knowledge base are formalized as constraints to modify the initial lesion propagation graph. Constraints can be hard constraints (must be satisfied) or soft constraints (satisfied as much as possible but with a certain deviation). Constraints are encoded and converted into attribute constraints, propagation probability constraints, etc. of nodes and edges in the graph structure. For example, a hard constraint can be expressed as the propagation probability of certain edges must be 0 or 1, and a soft constraint can be expressed as an adjustment suggestion or penalty item for the propagation probability.

[0105] Design a constraint correction algorithm to correct the initial lesion propagation map according to the constraints in the medical knowledge base. The algorithm steps may include: 1) Traverse all nodes and edges in the initial lesion propagation graph to check whether the constraints in the medical knowledge base are violated.

[0106] 2) For nodes or edges that violate constraints, take appropriate corrective measures based on the constraint type and severity, such as adjusting node attributes, modifying edge attributes, recalculating propagation probabilities, etc.

[0107] 3) During the correction process, balance the degree of constraint satisfaction and the coherence of the graph structure to avoid distortion of the graph structure caused by excessive correction.

[0108] The algorithm uses an iterative optimization approach to gradually adjust the graph structure until all hard constraints are met and the degree of satisfaction of soft constraints reaches an optimal or acceptable level.

[0109] After being processed by the constraint correction algorithm, a revised lesion propagation graph is generated. Based on the initial graph, this graph incorporates the constraints of the medical knowledge base and is more in line with the actual lesion propagation law and medical common sense. The revised lesion propagation graph is verified and evaluated to ensure its rationality and accuracy. The verification method includes comparing with the lesion propagation cases annotated by experts, calculating the connectivity index of the graph structure, and evaluating the rationality of the distribution of propagation probability.

[0110] S34: According to the corrected lesion propagation map, the lesion diffusion direction is obtained; the lesion diffusion direction is input into the pre-trained risk prediction model, and the symptom risk prediction result is output.

[0111] Specifically, the diffusion direction of the lesion is determined according to the corrected lesion propagation map. The diffusion direction can be determined by analyzing the direction of the edge in the graph and the propagation probability. The direction of the edge indicates the tendency of the lesion to propagate from one area to another, and the propagation probability indicates the strength of the tendency. For each lesion area (node), calculate its main diffusion direction, that is, the direction pointed by the edge with the highest propagation probability. For example, the main diffusion direction of node A is the direction of edge AB, and the propagation probability is P_AB. The main diffusion directions of all nodes are integrated to form an overall lesion diffusion direction map, which intuitively displays the main propagation paths and trends of lesions between different areas.

[0112] Design a visualization scheme for diffusion direction to show the lesion diffusion direction to users (such as doctors) in an intuitive way. For example, on a three-dimensional brain map, use arrows to indicate the main diffusion direction of each lesion area. The thickness of the arrow indicates the size of the spread probability, and the color indicates different lesion types or stages of spread. Provide explanations for the diffusion direction to help users understand the basis for determining the diffusion direction and its medical significance. For example, combine the rules in the medical knowledge base to explain the association between a certain diffusion direction and a specific lesion type or pathological mechanism.

[0113] The determined lesion diffusion direction is used as an input feature and input into the pre-trained symptom risk prediction model. The symptom risk prediction model can be a model based on machine learning or deep learning, such as logistic regression, random forest, neural network, etc., which is used to predict the risk of occurrence of different symptoms. The input feature dimension of the pre-trained symptom risk prediction model matches the representation of the lesion diffusion direction, and the output is the probability of occurrence of different symptoms. For example, the model outputs a probability vector, each element of which corresponds to the probability of occurrence of a symptom, such as the probability of headache is 0.7, the probability of limb weakness is 0.3, etc.

[0114] The symptom risk prediction model generates symptom risk prediction results based on the input lesion diffusion direction. The results include the probability of occurrence of different symptoms, risk levels (such as high risk, medium risk, low risk), and possible time ranges for symptom onset. The symptom risk prediction results are applied to clinical decision support to provide doctors with symptom risk assessments for patients and assist in formulating treatment plans and intervention measures. For example, for high-risk symptoms, preventive treatment or close monitoring is recommended in advance; for low-risk symptoms, the frequency of examinations can be appropriately reduced to optimize the allocation of medical resources.

[0115] refer to Figure 5 In an optional embodiment, S4 includes the following steps: S41: Load the lesion diffusion direction in the 3D brain model and generate dynamic arrow trajectories.

[0116] Specifically, a standardized 3D brain model is obtained, such as a 3D brain structure model based on a brain map template (such as an MNI template). The model represents the anatomical structure of the brain in the form of a 3D grid or voxel, and contains information on different brain regions, lobes, etc. The 3D brain model is preprocessed, including coordinate system 1 and resolution adjustment, so that it matches the coordinate system and resolution of the neurology image data to ensure that the direction of lesion diffusion can be accurately loaded into the model.

[0117] The lesion diffusion direction data obtained in S3 is loaded into the 3D brain model. The diffusion direction data includes information such as the diffusion direction vector and the starting position coordinates of each lesion area. The diffusion direction data is matched with the corresponding position in the 3D brain model through a coordinate matching algorithm. For example, according to the position coordinates of the lesion area, the corresponding brain area position is found in the 3D brain model, and the diffusion direction vector is associated with the position.

[0118] Design a dynamic arrow trajectory generation algorithm to generate intuitive arrow trajectories in the 3D brain model based on the lesion diffusion direction data. The algorithm steps may include: 1) Determine the starting and ending points of the arrow based on the diffusion direction vector and the starting position coordinates.

[0119] 2) Calculate the shape and size of the arrows so that they can clearly indicate the diffusion direction and intensity. For example, the length of the arrow can indicate the diffusion distance, and the thickness of the arrow can indicate the diffusion probability or intensity.

[0120] 3) Generate dynamic effects of arrows, such as gradient display and animation playback of arrows, to show the diffusion process of lesions over time.

[0121] Mathematical methods such as Bezier curves or spline curves are used to make the arrow trajectory transition smoothly in 3D space, enhancing the smoothness of the visual effect.

[0122] In the 3D brain model, dynamic arrow trajectories are displayed in an overlay manner. Arrow trajectories can be rendered in semi-transparent or different colors to distinguish them from the anatomical structure of the brain model and avoid blocking important information. Interactive control functions are provided, such as adjusting the transparency, color, and animation speed of the arrows, to facilitate doctors to observe and analyze as needed. For example, doctors can pause the animation at a certain point in time to view the spread of the lesion in detail.

[0123] S42: Perform color gradient overlay processing on the 3D brain model based on the symptom risk prediction results and mark the high-risk areas.

[0124] Specifically, a color gradient scheme is designed to indicate the risk levels of different symptoms. The color gradient from low risk to high risk can be a gradual change from blue (low risk) to red (high risk), with green, yellow and other colors in between to indicate the gradual increase in risk. Define the mapping relationship between color gradient and risk value, and map the probability value or risk level in the symptom risk prediction result to the corresponding color value. For example, risk values ​​between 0-0.3 are mapped to the blue series, between 0.3-0.7 are mapped to the yellow series, and between 0.7-1.0 are mapped to the red series.

[0125] According to the symptom risk prediction results, high-risk areas in the 3D brain model are determined. High-risk areas can be brain regions associated with specific high-risk symptoms, or areas with higher risk accumulation on the path of lesion diffusion. Color gradient overlay processing is performed on high-risk areas, and the corresponding color values ​​are applied to the corresponding positions of the brain model. For example, for a certain high-risk brain area, the color of its voxel or mesh patch in the 3D brain model is replaced with the color calculated according to the risk value.

[0126] Image processing and computer graphics techniques are used to implement color gradient superposition on the 3D brain model. Texture mapping, voxel coloring and other methods can be used to combine color information with the geometric data of the brain model. The visual fusion problem in the color superposition process is handled to ensure that the superimposed color and the original color of the brain model transition naturally without causing obvious visual conflicts. For example, by adjusting the transparency of the color, the fusion algorithm, etc., the color of the high-risk area can be highlighted on the brain model without covering up the anatomical structure information.

[0127] The visual effects of the 3D brain model after superimposing colors are optimized, such as adjusting parameters such as lighting, shadows, and reflections to enhance the three-dimensional and layered sense of the model, making high-risk areas more intuitive and visible. Interactive functions are provided, such as switching color gradient display, viewing areas with specific risk levels, and querying the risk value of a certain point, so that doctors can conduct in-depth analysis of symptom risk distribution. For example, doctors can hover the mouse over a certain brain area to pop up specific risk information and related symptom prompts for that area.

[0128] S43: Generate a dynamic lesion indication map based on the dynamic arrow trajectory and the high-risk area.

[0129] Specifically, a multi-layer structure of the dynamic lesion indication map is constructed, including the basic brain model layer, the lesion diffusion arrow trajectory layer, the symptom risk color overlay layer, etc. Each layer manages its data and display properties independently, which is convenient for individual adjustment and update. The layer control interface is designed to allow doctors to choose to display or hide a layer as needed, and adjust the layer's order and transparency and other properties. For example, doctors can hide the lesion diffusion arrow trajectory layer and only view the symptom risk color overlay layer, or increase the transparency of the basic brain model layer to highlight the information of other layers.

[0130] Integrate the data of each layer to ensure that the information of each layer remains synchronized and consistent during dynamic changes. For example, when the lesion diffusion arrow trajectory is updated, the color and risk value of the relevant area in the symptom risk color overlay layer are adjusted accordingly to ensure the coordination and accuracy of the entire dynamic lesion indication map. Establish a data update mechanism to automatically trigger the update of the layer data when a new lesion diffusion direction or symptom risk prediction result is generated, reflecting the latest disease analysis results in real time.

[0131] Choose a suitable rendering engine, such as a renderer based on OpenGL, Vulkan and other graphics APIs, or use a professional medical image visualization software library (such as ITK-SNAP, 3D Slicer, etc.) to render the dynamic lesion indication map. Optimize the rendering process to improve rendering speed and image quality. For example, use multi-threaded rendering technology to make full use of the computer's multi-core processor resources; use texture compression, geometry simplification and other methods to reduce the computational complexity of rendering, and ensure that the dynamic lesion indication map can smoothly display animation effects.

[0132] Determine the output format of the dynamic lesion indication map, and select the appropriate format according to different application scenarios. For example, for clinical diagnosis systems, it can be output as an interactive 3D model file (such as .obj, .stl format) and embedded in medical software; for scientific research reports or patient education, it can be output as a dynamic video file (such as .gif, .mp4 format) to intuitively display the dynamic changes and risk distribution of lesions. Perform quality inspection and verification on the output dynamic lesion indication map to ensure that the accuracy, completeness and visual effects of the image meet the requirements. For example, check whether the arrow trajectory is complete, whether the color overlay is correct, whether the layer display is normal, etc., to avoid misjudgment or misunderstanding due to rendering or output problems.

[0133] S44: In response to the correction path input by the doctor, the dynamic arrow trajectory is optimized through the adversarial generative network to update the dynamic lesion indication map.

[0134] Specifically, a generative adversarial network (GAN) is constructed, including two sub-networks: a generator and a discriminator. The input of the generator is the lesion diffusion path corrected by the doctor and the original dynamic arrow trajectory data, and the output is the optimized dynamic arrow trajectory. The generator can adopt an architecture based on a convolutional neural network (CNN) or a recurrent neural network (RNN) and be designed according to the characteristics of the path data. For example, for spatial path data, CNN is used to capture spatial features; for time series path data, RNN is used to model time dependencies. The input of the discriminator is the optimized dynamic arrow trajectory and the real lesion diffusion path data, and the output is the judgment probability of the authenticity of the trajectory. The discriminator can also adopt a CNN or RNN architecture to form a closed loop of adversarial training with the generator.

[0135] Define the loss function of GAN, including the loss of the generator and the loss of the discriminator. The loss of the generator aims to make the generated optimization path as close to the real path as possible while satisfying the constraints of the doctor's correction; the loss of the discriminator aims to accurately distinguish the generated path from the real path. Use an optimization algorithm (such as the Adam optimizer) to train GAN. By alternately optimizing the generator and the discriminator, the entire network reaches a Nash equilibrium state, and the generator can generate high-quality optimization paths.

[0136] Receive the corrected path input by the doctor, which can be the result of the doctor's manual adjustment of the initial dynamic arrow trajectory based on clinical experience, including the doctor's professional judgment and correction information on the direction of lesion spread. Perform data preprocessing on the doctor's corrected path, including coordinate normalization, path smoothing, etc., to match it with the input requirements of GAN. For example, normalize the corrected path coordinates to the range of [0,1] to remove noise points and mutation points in the path.

[0137] The doctor's corrected path and the original dynamic arrow trajectory data are input into the GAN generator, which generates a preliminary optimized path based on these data. The generated optimized path is input into the discriminator, which evaluates its similarity with the true path and passes the feedback information to the generator. The generator adjusts the network parameters based on the feedback information and continuously optimizes the generated path to make the generated path closer to the actual situation and meet the doctor's correction intention. Repeat the above process for multiple rounds until the generated optimized path converges on the loss function or meets the preset optimization criteria, such as the smoothness of the path, the similarity with the true path, and the compliance with the doctor's correction.

[0138] Apply the optimized dynamic arrow trajectory to the dynamic lesion indication map, replace the original arrow trajectory data, and update the layer display. At the same time, re-evaluate the symptom risk distribution according to the optimized path, and update the symptom risk color overlay layer accordingly if necessary. Verify and evaluate the updated dynamic lesion indication map to ensure that the optimized path is reasonable in medical logic, clear and accurate in visual effects, and consistent with the doctor's revised intention. Figure 1 Verification methods can include comparing with the standard path marked by experts, calculating the error index before and after the path optimization, and inviting doctors to conduct subjective evaluation.

[0139] refer to Figure 6 In an optional embodiment, S44 includes the following steps: S441: Use the corrected path as the input of the generator in the adversarial generative network to generate pseudo propagation path data.

[0140] Specifically, the lesion diffusion path corrected by the doctor is used as the input data of the generator in the adversarial generative network. The corrected path can be represented as a series of coordinate points or vectors, describing the expected diffusion direction and path of the lesion in the 3D brain model. The input data is preprocessed, including normalization and dimension adjustment, to meet the input requirements of the generator. For example, the path coordinates are normalized to the range of [0,1] to ensure the numerical stability and consistency of the data.

[0141] The generator uses its internal neural network structure (such as CNN or RNN) to extract features and generate paths based on the input correction path. The network parameters of the generator are randomly initialized and need to be optimized through the subsequent training process. The generated pseudo propagation path data has the same dimension and format as the input correction path, but there may be differences in details and shape because the generator attempts to simulate the real lesion propagation pattern.

[0142] S442: The pseudo propagation path data is compared and verified with the real pathological propagation pattern through the discriminator in the adversarial generative network to optimize the parameters of the generator.

[0143] Specifically, the input of the discriminator consists of two parts: one is the pseudo propagation path data generated by the generator, and the other is the real propagation path data sampled from the real pathological propagation pattern. The real pathological propagation pattern data can be obtained from the historical case database, annotated and verified, and used as the benchmark for training the discriminator. The real propagation path data is preprocessed, matched and aligned with the pseudo propagation path data to ensure that the two are consistent in dimension and format, which is convenient for the discriminator to perform comparative analysis.

[0144] The discriminator extracts features and classifies the input propagation path data through its internal neural network structure (such as multi-layer perceptron MLP or CNN). The goal of the discriminator is to accurately distinguish between false propagation paths and real propagation paths, and output the true probability value of each path data. During the training process of the discriminator, the loss function (such as binary cross entropy loss) is used to measure the accuracy of its judgment, and the network parameters are adjusted through the back-propagation algorithm to optimize the performance of the discriminator.

[0145] The loss function of the generator is defined to make the generated pseudo propagation path as close as possible to the real propagation path while considering the constraints of the doctor's correction path. The loss function can include the following parts: 1) Adversarial loss: measures the similarity between the path generated by the generator and the real path under the judgment of the discriminator, prompting the generator to generate more realistic paths.

[0146] 2) Reconstruction loss: measures the difference between the generated path and the doctor’s correction path to ensure that the generated path does not deviate from the doctor’s professional correction intention.

[0147] 3) Regularization loss: To prevent the generator from overfitting, add L2 regularization terms to constrain the size of network parameters.

[0148] The generator is trained using an optimization algorithm (such as the Adam optimizer) and the network parameters of the generator are adjusted by minimizing the loss function. The training process uses an alternating training method: first, the generator parameters are fixed and the discriminator is trained for several rounds so that the discriminator can accurately distinguish between real and fake paths; then the discriminator parameters are fixed and the generator is trained for several rounds so that the generator can generate more realistic paths. The above process is repeated until the generator and the discriminator reach a Nash equilibrium state, and the path generated by the generator is indistinguishable from the real path in the eyes of the discriminator.

[0149] S443: Update the propagation path output by the optimized generator to the dynamic lesion indication map.

[0150] Specifically, the propagation path output by the optimized generator is post-processed, including smoothing, denoising, and coordinate denormalization. Smoothing can use, for example, a Savitzky-Golay filter to remove subtle fluctuations and noise points in the path, making the path smoother and more natural. Coordinate denormalization converts the path coordinates from the normalized range back to the coordinate system of the original 3D brain model to ensure that the path accurately matches the anatomical structure of the brain model.

[0151] Verify whether the optimized propagation path is consistent with medical logic and clinical reality. The verification method includes comparing with the lesion propagation rules in the medical knowledge base to check whether the path violates the known pathological propagation pattern; invite medical experts to conduct subjective evaluation of the generated path to judge its rationality and credibility.

[0152] In the multi-layer structure of the dynamic lesion indication map, find the arrow trajectory layer related to the lesion diffusion direction and update the optimized propagation path data to this layer. Replace the original arrow trajectory data to ensure that the path displayed by the layer is the latest optimized result. At the same time, based on the optimized path, re-evaluate the distribution of symptom risks related to it, and update the symptom risk color overlay layer accordingly if necessary to keep the entire dynamic lesion indication map coordinated and consistent.

[0153] Trigger the rendering update process of the dynamic lesion indication map and regenerate the image or animation containing the optimized propagation path. During the rendering process, ensure that the fusion and display effects between the layers meet the expectations, such as the accurate matching of the arrow trajectory with the anatomical structure of the brain model, the natural transition of color overlay, etc. Show the updated dynamic lesion indication map to the doctor to facilitate further analysis and diagnosis. Provide interactive functions, such as playback control of path animation and layer display switching, to enhance the doctor's understanding and grasp of the spread of lesions.

[0154] In an optional embodiment, the S1 includes the following steps: S11: Perform time series registration processing on multi-phase neurology image data to generate a time-aligned image sequence.

[0155] Specifically, for multi-phase neurological image data, feature-based registration algorithms such as SIFT (Scale Invariant Feature Transform) or SURF (Speeded Up Robust Features) algorithms are selected. These algorithms can effectively extract key feature points in the image and match between images at different time points. They are suitable for processing neurological images that may have changes in position, angle, and scale. For situations where high-precision registration is required, a multimodal image registration algorithm combined with the mutual information criterion is used. The mutual information criterion can measure the statistical dependence between images of different modalities and is suitable for registration between MRI and CT images, improving the accuracy and robustness of the registration.

[0156] Preprocess the multi-phase neurology image data of each patient, including noise removal, contrast adjustment and other operations to improve image quality and enhance feature detectability. Use the selected registration algorithm to extract and match feature points of the image. For example, the SIFT algorithm is used to detect scale-invariant feature points in the image, and then the RANSAC (random sampling consensus) algorithm is used to estimate the geometric transformation model (such as affine transformation, thin plate spline transformation) from the matched feature point pairs to achieve time series registration of the image. Post-process the registered image, such as interpolation to fill the hole area generated during the registration process, to ensure the integrity and continuity of the image.

[0157] A variety of indicators are used to evaluate the quality of registration, including the accuracy and recall of feature point matching, mutual information value, root mean square error (RMS), etc. For example, RMS is calculated by comparing the position deviation of feature points before and after registration. The smaller the RMS value, the higher the registration accuracy. Visual inspection is performed on the registered images to observe the alignment of anatomical structures, such as whether the sulci and gyri are correctly aligned, to determine whether the registration effect meets expectations.

[0158] According to the quality assessment results, the registration algorithm is optimized and adjusted. For example, if the feature point matching accuracy is low, the parameters of the feature detection algorithm can be adjusted, such as increasing the feature detection threshold of the SIFT algorithm to improve the quality and quantity of feature points; if the mutual information value does not meet expectations, different registration models can be tried or model parameters can be adjusted, such as using a more complex nonlinear transformation model.

[0159] S12: performing grayscale normalization processing on the time-aligned image sequence to obtain standardized image data.

[0160] Specifically, according to the characteristics of neurology image data, a suitable grayscale normalization method is selected. For modalities with relatively stable image grayscale value distribution (such as T1-weighted MRI), a linear normalization method is used to map the grayscale value to a preset range (such as [0,255] or [0,1]). For image data with complex grayscale value distribution or abnormal values ​​(such as areas with large grayscale value deviations in certain pathological conditions), nonlinear normalization methods such as logarithmic normalization or piecewise linear normalization are used to better adjust the grayscale value distribution and enhance the contrast and details of the image.

[0161] Determine the parameters of grayscale normalization, such as the target grayscale range, parameters of the nonlinear transformation function, etc. For linear normalization, calculate the original grayscale range (minimum and maximum values), and determine the linear mapping relationship based on the target grayscale range. Perform grayscale normalization on each image in the time-aligned image sequence, and convert each pixel value in the image matrix to the target grayscale range. For example, for linear normalization, the new value of the normalized pixel value = (original pixel value - original minimum value) / (original maximum value - original minimum value) × (target maximum value - target minimum value) + target minimum value.

[0162] The grayscale normalization effect is evaluated by combining statistical analysis and visual evaluation. In terms of statistical analysis, the grayscale histogram of the normalized image is observed to check whether it conforms to the expected distribution form, such as whether it is evenly distributed or conforms to a certain probability distribution; in terms of visual evaluation, the visual effects of the image before and after normalization are compared to determine whether the image clarity and contrast are enhanced. Expert evaluation is conducted on some images, and medical imaging experts are invited to score and evaluate the quality of the normalized images, collect feedback, and understand whether the normalization process meets the needs of clinical diagnosis and analysis.

[0163] Formulate adjustment strategies based on the effect evaluation results. If abnormal peaks or valleys appear in the grayscale histogram, it may be necessary to reselect the normalization method or adjust the parameters, such as changing linear normalization to nonlinear normalization, or adjusting the target grayscale range; if the visual effect is not good, such as the image is too dark or too bright, the normalized image can be post-processed by adjusting the brightness and contrast. Re-evaluate the effect of the adjusted normalization method and parameters until a satisfactory normalization effect is achieved to ensure that the quality of the standardized image data meets the requirements of subsequent analysis.

[0164] S13: Perform time-series alignment processing on the symptom records and medication time points in the clinical data and the standardized imaging data to generate a standardized spatiotemporal dataset.

[0165] Specifically, a unified time reference point is determined in clinical data and standardized imaging data. Usually, the time of the patient's first visit or the first imaging examination is used as the time reference point (Time Zero). For example, the MRI examination time when the patient is first diagnosed is set as Time Zero, and the time points of all subsequent imaging and clinical data are recorded relative to this time point. Other important clinical event time points (such as surgery time, treatment start time, etc.) are also clearly recorded and labeled so that relevant imaging and clinical information can be accurately associated during the time alignment process.

[0166] Formulate data association rules to associate symptom records and medication time points in clinical data with standardized imaging data. The basis for association includes key information such as patient ID and timestamp to ensure that each symptom record and medication time point can accurately correspond to the corresponding imaging data time point. Establish data association tables or database relationships to integrate imaging data, symptom records, medication time points and other information into a unified spatiotemporal data structure. For example, create a data table in the database that contains fields such as patient ID, imaging time point, imaging data path, symptom record content, medication time point and medication dosage to achieve centralized data management and convenient query.

[0167] In the process of time series alignment, an accurate time matching algorithm is used, taking into account the format and precision of timestamps. For example, all timestamps are uniformly converted to a standard time format (such as ISO 8601 format) and retained to the second level or finer precision units to avoid alignment errors caused by inconsistent time formats or insufficient precision. For data with time deviations or inaccurate records, data cleaning and correction are performed. For example, by cross-validating other data sources (such as medical records, nursing records, etc.) to correct incorrect time point records, ensure that the time information of the data is authentic and reliable.

[0168] Integrate the image data, symptom records and medication time points that have completed time alignment into a standardized spatiotemporal data set to form a complete and ordered data set, providing a unified data basis for subsequent analysis and processing. Perform comprehensive verification and inspection on the integrated standardized spatiotemporal data set to ensure the integrity and accuracy of the data. The verification content includes whether the number of data records matches, whether the time sequence is reasonable, and whether the data format meets the requirements. For example, check whether each time point has corresponding image data and clinical data, whether the symptom records and medication time points are within a reasonable time range, and whether the format of the data file is consistent with the preset format.

[0169] The above-mentioned image-assisted data processing method suitable for neurology generates a spatiotemporal data set through time series alignment and standardization of multi-phase neurology image data and clinical data, uses a 4D convolutional network to extract dynamic features of lesions and combines a multi-head attention mechanism to construct a spatiotemporal symptom association, uses a graph neural network to simulate the lesion propagation path and integrates the medical knowledge base to constrain and optimize the prediction results, generates a dynamic visualization indicator map to support doctor-patient collaborative correction, and finally realizes model adaptive iteration through incremental learning and closed-loop verification mechanism, realizes dynamic tracking of neurological lesions, propagation path prediction and symptom risk quantitative assessment, significantly improves the time series analysis accuracy of disease evolution and the interpretability of prediction results, and at the same time enhances the personalization and reliability of clinical decision-making through multimodal data fusion and interactive correction mechanism.

[0170] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0171] Based on the same inventive concept, the embodiment of the present application also provides a system for implementing the above-mentioned method for image-assisted data processing applicable to neurology. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the system for image-assisted data processing applicable to neurology provided below can refer to the above-mentioned limitations on a method for image-assisted data processing applicable to neurology, and will not be repeated here.

[0172] In an exemplary embodiment, Figure 7 As shown, a neurology image-assisted data processing system 70 is provided, comprising: The data processing module 71 is used to generate a standardized spatiotemporal data set based on the multi-phase neurological image data and clinical data of the patient.

[0173] The feature analysis module 72 is used to extract the dynamic features of the lesion area based on the standardized spatiotemporal data set; based on the dynamic features, construct a spatiotemporal symptom association matrix.

[0174] The prediction and analysis module 73 is used to predict the lesion propagation path of the lesion area based on the spatiotemporal symptom association matrix and use the lesion propagation path prediction model to obtain the lesion diffusion direction and symptom risk prediction results.

[0175] The visualization module 74 is used to generate a dynamic lesion indication map based on the lesion diffusion direction and symptom risk prediction results.

[0176] The optimization module 75 is used to iteratively optimize the lesion propagation path prediction model by comparing the symptom risk prediction results with the clinical data.

[0177] Optionally, the feature analysis module 72 includes: The spatiotemporal feature extraction unit is used to perform spatiotemporal feature extraction processing on the standardized spatiotemporal data set based on a 4D convolutional neural network to obtain the dynamic features of the lesion area; the dynamic features include volume change features, position coordinate change features, and diffusion speed change features.

[0178] The feature fusion unit is used to perform feature fusion processing on the dynamic features and the symptom records in the clinical data to generate a first correlation matrix.

[0179] The weight allocation unit is used to perform weight allocation processing on the correlation relationship between the lesion area and the symptoms in the first correlation matrix through a multi-head attention mechanism, and obtain a second correlation matrix containing the correlation weights between the lesion area and the symptoms as a spatiotemporal symptom correlation matrix.

[0180] Optionally, the spatiotemporal feature extraction unit includes: The spatial feature extraction subunit is used to extract the lesion structure features by performing 3D convolution processing on the standardized spatiotemporal dataset in the spatial dimension.

[0181] The temporal feature extraction subunit is used to model the lesion evolution trend by processing the standardized spatiotemporal dataset using an LSTM network in the temporal dimension.

[0182] The data fusion subunit is used to perform spatiotemporal fusion processing on the lesion structural characteristics and lesion evolution trends to generate a dynamic lesion evolution map.

[0183] The feature generation subunit is used to extract the volume change characteristics, position coordinate change characteristics and diffusion speed change characteristics of the lesion area from the dynamic lesion evolution map.

[0184] Optionally, the prediction analysis module 73 includes: The graph structure construction unit is used to construct an initial graph structure with the lesion area as the node and the diffusion path as the edge; the attributes of the node include the volume, position coordinates and diffusion speed of the lesion area, and the attributes of the edge include the diffusion direction and time interval; the lesion propagation path prediction model includes the initial graph structure and the graph attention network.

[0185] The propagation probability calculation unit is used to input the association weights in the spatiotemporal symptom association matrix as node attributes into the initial graph structure, calculate the propagation probability between nodes through the message passing mechanism of the graph attention network, and generate an initial lesion propagation graph containing the propagation weights.

[0186] The propagation map correction unit is used to perform constraint correction processing on the lesion propagation map based on the lesion propagation rules predefined in the medical knowledge base to obtain a corrected lesion propagation map.

[0187] The diffusion direction and risk prediction unit is used to obtain the lesion diffusion direction according to the corrected lesion propagation map; input the lesion diffusion direction into the pre-trained risk prediction model, and output the symptom risk prediction result.

[0188] Optionally, the visualization module 74 includes: The diffusion direction visualization unit is used to load the lesion diffusion direction in the 3D brain model and generate dynamic arrow trajectories.

[0189] The risk area labeling unit is used to perform color gradient overlay processing on the 3D brain model according to the symptom risk prediction results and label the high-risk areas.

[0190] The indication map generating unit is used to generate a dynamic lesion indication map based on the dynamic arrow trajectory and the high-risk area.

[0191] The path optimization unit is used to respond to the corrected path input by the doctor, perform path optimization processing on the dynamic arrow trajectory through the adversarial generative network, and update the dynamic lesion indication map.

[0192] Optionally, the path optimization unit includes: The pseudo data generation subunit is used to use the corrected path as the input of the generator in the adversarial generation network to generate pseudo propagation path data.

[0193] The parameter optimization subunit is used to compare and verify the pseudo propagation path data with the actual pathological propagation pattern through the discriminator in the adversarial generation network, and optimize the parameters of the generator.

[0194] The indication map updating subunit is used to update the propagation path output by the optimized generator to the dynamic lesion indication map.

[0195] Optionally, the data processing module 71 includes: The time series registration unit is used to perform time series registration processing on multi-phase neurology image data to generate a time-aligned image sequence.

[0196] The grayscale normalization unit is used to perform grayscale normalization processing on the time-aligned image sequence to obtain standardized image data.

[0197] The time series alignment unit is used to perform time series alignment processing on the symptom records and medication time points in the clinical data with the standardized imaging data to generate a standardized spatiotemporal data set.

[0198] An embodiment of the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0199] The embodiments of the present application further provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0200] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0201] The above-mentioned embodiments only express several implementation methods of the embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the embodiments of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the embodiments of the present application, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A method for image-assisted data processing applicable to neurology, characterized in that: The method comprises: S1: Generate a standardized spatiotemporal dataset based on the patient's multi-phase neurological image data and clinical data; S2: extracting dynamic features of the lesion area based on the standardized spatiotemporal data set; constructing a spatiotemporal symptom association matrix based on the dynamic features; S3: Based on the spatiotemporal symptom association matrix, a lesion propagation path prediction model is used to predict the lesion propagation path of the lesion area to obtain a lesion diffusion direction and symptom risk prediction result; S4: generating a dynamic lesion indication map based on the lesion diffusion direction and symptom risk prediction results; S5: Iteratively optimize the lesion propagation path prediction model by comparing the symptom risk prediction results with the clinical data.

2. The method according to claim 1, characterized in that: The S2 includes: S21: performing spatiotemporal feature extraction processing on the standardized spatiotemporal data set based on a 4D convolutional neural network to obtain the dynamic features of the lesion area; the dynamic features include volume change features, position coordinate change features, and diffusion velocity change features; S22: performing feature fusion processing on the dynamic features and the symptom records in the clinical data to generate a first correlation matrix; S23: Perform weight assignment processing on the correlation between the lesion area and the symptoms in the first correlation matrix through a multi-head attention mechanism to obtain a second correlation matrix containing the correlation weights between the lesion area and the symptoms as the spatiotemporal symptom correlation matrix.

3. The method according to claim 2, characterized in that The S21 includes: S211: extracting lesion structure features by performing 3D convolution processing on the standardized spatiotemporal dataset in the spatial dimension; S212: Modeling the lesion evolution trend by performing LSTM network processing on the standardized spatiotemporal dataset in the time dimension; S213: performing spatiotemporal fusion processing on the lesion structure characteristics and lesion evolution trends to generate a dynamic lesion evolution map; S214: Extracting the volume change characteristics, the position coordinate change characteristics, and the diffusion speed change characteristics of the lesion area from the dynamic lesion evolution map.

4. The method according to claim 2, characterized in that: The S3 includes: S31: constructing an initial graph structure with the lesion area as nodes and the diffusion path as edges; wherein the attributes of the nodes include the volume, position coordinates and diffusion speed of the lesion area, and the attributes of the edges include the diffusion direction and time interval; the lesion propagation path prediction model includes the initial graph structure and a graph attention network; S32: inputting the association weights in the spatiotemporal symptom association matrix as node attributes into the initial graph structure, calculating the propagation probabilities between the nodes through the message passing mechanism of the graph attention network, and generating an initial lesion propagation graph including the propagation weights; S33: performing constraint correction processing on the lesion propagation graph based on the lesion propagation rules predefined in the medical knowledge base to obtain a corrected lesion propagation graph; S34: Obtain the lesion diffusion direction according to the corrected lesion propagation map; input the lesion diffusion direction into a pre-trained risk prediction model, and output the symptom risk prediction result.

5. The method according to claim 1, characterized in that: The S4 includes: S41: loading the lesion diffusion direction in the 3D brain model to generate a dynamic arrow trajectory; S42: performing color gradient superposition processing on the 3D brain model according to the symptom risk prediction result, and marking high-risk areas; S43: generating the dynamic lesion indication map based on the dynamic arrow trajectory and the high-risk area; S44: In response to the correction path input by the doctor, the dynamic arrow trajectory is subjected to path optimization processing by a generative adversarial network, and the dynamic lesion indication map is updated.

6. The method according to claim 5, characterized in that The S44 includes: S441: Using the corrected path as an input of a generator in the generative adversarial network to generate pseudo propagation path data; S442: Compare and verify the pseudo propagation path data with the real pathological propagation mode through the discriminator in the adversarial generative network, and optimize the parameters of the generator; S443: Update the optimized propagation path output by the generator to the dynamic lesion indication map.

7. The method according to any one of claims 1 to 6, characterized in that The S1 includes: S11: performing time series registration processing on the multi-phase neurology image data to generate a time-aligned image sequence; S12: performing grayscale normalization processing on the time-aligned image sequence to obtain standardized image data; S13: Performing time-series alignment processing on the symptom records and medication time points in the clinical data and the standardized image data to generate the standardized spatiotemporal data set.

8. An image-assisted data processing system suitable for neurology, characterized in that: The system comprises: A data processing module, used for generating a standardized spatiotemporal data set based on the multi-phase neurological image data and clinical data of the patient; A feature analysis module, used to extract dynamic features of the lesion area based on the standardized spatiotemporal data set; and to construct a spatiotemporal symptom association matrix based on the dynamic features; The prediction analysis module is used to predict the lesion propagation path of the lesion area based on the spatiotemporal symptom association matrix and the lesion propagation path prediction model to obtain the lesion diffusion direction and symptom risk prediction results; A visualization module, used for generating a dynamic lesion indication map based on the lesion diffusion direction and symptom risk prediction results; An optimization module is used to iteratively optimize the lesion propagation path prediction model by comparing the symptom risk prediction results with the clinical data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images