A multi-modal fusion cerebral infarction early warning system and method

Abstract

The application relates to the technical field of medical data analysis, and particularly discloses a multi-modal fusion cerebral infarction early warning system and method, which acquires imaging data, serological data and clinical scale data; a heterogeneity feature set of embedding space position base and time evolution base is generated through joint transformation of variational modal decomposition and random Fourier feature mapping; a sparse Gaussian process is used to perform heterogeneity feature decoupling in a function space, to output pathological stable features and physiological non-specific noise features, and to calculate cognitive uncertainty; the pathological stable features are dynamically weighted and reconstructed according to the cognitive uncertainty, and are mapped to a shared high-dimensional semantic space to form multi-modal joint feature representation; the multi-modal joint feature representation is input into a Bayesian time series prediction framework, an evolution trend is analyzed, and a cerebral infarction super-early warning probability is output; the application can effectively distinguish pathological features from non-specific interference signals, reduce the false positive rate, and accurately warn of the super-early state of cerebral infarction.

Description

Technical Field

[0001] This invention relates to the field of medical data analysis technology, specifically to a multimodal fusion early warning system and method for cerebral infarction. Background Technology

[0002] Cerebral infarction is characterized by high incidence, high disability rate, and high mortality rate. Ultra-early intervention is crucial for improving patient prognosis. Currently, commonly used clinical risk assessments for cerebral infarction mainly rely on single-modality data, such as imaging examinations, serological markers, or clinical scale assessments. While diffusion-weighted imaging sequences can visualize acute ischemic lesions, in the ultra-early stages, the lesion signal is weak and the spatial proportion is extremely small, making it easy to miss. Serological markers such as high-sensitivity C-reactive protein and S100 calcium-binding protein B are related to the pathological process of cerebral infarction, but they are greatly affected by non-specific factors such as inflammation and stress, and their use alone can easily produce false positives. Clinical scale assessments, such as the National Institutes of Health Stroke Scale (NIHSS), are affected by the subjective factors of the assessor and the patient's cooperation, making it difficult to objectively reflect ultra-early pathological changes.

[0003] How can we accurately identify and separate stable feature signals driven by the pathological evolution of ultra-early cerebral infarction, which are weakly distributed in numerical expression and occupy a very small spatial area, from interference signals generated by non-specific physiological stress responses, which are significantly fluctuating in numerical expression but lack pathological specificity, in a high-dimensional heterogeneous feature space? Furthermore, how can we capture the dynamic geometric features of the transition from latent accumulation to explicit transformation of pathological state in the continuous time function space of multimodal data temporal co-evolution, so as to overcome the "pseudo-attention" focusing defect caused by the cross-modal semantic gap in existing technologies, avoid misjudging transient, high-amplitude physiological parameter fluctuations as early warning signs, and thus achieve accurate characterization of the pathological essence of ultra-early cerebral infarction? Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal fusion early warning system and method for cerebral infarction to solve the problems mentioned above.

[0005] The objective of this invention can be achieved through the following technical solutions: A multimodal fusion method for early warning of cerebral infarction includes the following steps: S1, Acquire multimodal data, which includes imaging data, serological data, and clinical scale data; S2, apply a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; S3, based on the heterogeneous feature set, uses a sparse Gaussian process to perform heterogeneous feature decoupling in the function space, fits the local structural stability and global volatility of each modality data respectively, outputs the decoupled pathological stability features and physiological nonspecific noise features, and calculates the cognitive uncertainty corresponding to the pathological stability features. S4. Based on cognitive uncertainty, the pathological stable features are dynamically reconstructed with weights, and the pathological stable features are mapped to a shared high-dimensional semantic space to form a multimodal joint feature representation with consistency in pathological evolution. S5 inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework, and outputs the probability of ultra-early warning of cerebral infarction by analyzing the evolution trend of the multimodal joint feature representation in time.

[0006] As a further aspect of the present invention: S2 specifically includes: A three-dimensional topological graph structure is constructed based on the spatial neighborhood relationship of imaging data. Variational mode decomposition is used to decompose the voxel intensity sequence in the three-dimensional topological graph structure into several intrinsic mode functions. The intrinsic mode components corresponding to the changes in tissue microstructure are extracted as spatial location basis. For serological data and clinical scale data, time series were constructed respectively. The transient fluctuations and continuous evolution trends in the time series were separated by variational mode decomposition, and the intrinsic mode components that characterize the inertia of pathological evolution were extracted as the time evolution basis. The spatial location basis and the temporal evolution basis are respectively mapped to a high-dimensional regenerating kernel Hilbert space through random Fourier feature mapping. The two types of basis are then subjected to Hadamard product operation in the high-dimensional regenerating kernel Hilbert space to generate a heterogeneous feature set that embeds the interaction relationship between the spatial location basis and the temporal evolution basis.

[0007] As a further aspect of the present invention: S3 specifically includes: The decoupled pathological stability features and physiological nonspecific noise features are projected onto the regeneration kernel Hilbert space respectively, and the variance distribution parameters of the pathological stability features in the regeneration kernel Hilbert space are extracted. Based on the fluctuation amplitude of physiological nonspecific noise characteristics in the function space, a local energy spectral density is constructed to characterize the degree of signal distortion. The ratio between the variance distribution parameter of pathological stable features and the local energy spectral density of physiological nonspecific noise features is used as an uncertainty quantification factor. The cognitive uncertainty is output by performing a nonlinear normalization transformation through the comparison value.

[0008] As a further aspect of the present invention: the construction of the local energy spectral density for characterizing the degree of signal distortion specifically includes: Extract the instantaneous angular frequency of physiological nonspecific noise features in the function space, and determine the fluctuation irregularity at each sampling point based on the gradient change of the instantaneous angular frequency; Based on the volatility irregularity, a phase consistency function is constructed in the function space. The phase consistency function is used to distinguish between structural volatility and random disturbance. Within a local neighborhood of the phase coherence function space, energy is weighted and aggregated according to the phase coherence function to obtain a local energy spectral density that reflects the degree of signal distortion.

[0009] As a further aspect of the present invention: the construction of the phase consistency function specifically includes: Based on the fluctuation irregularity, the instantaneous phase at each sampling point in the function space is determined, and the instantaneous phase is mapped onto the unit circle of the complex plane to obtain the complex phase characterization; Within a local neighborhood of the function space, the complex phase representations are superimposed to obtain the resultant amplitude and resultant phase angle of the superimposed vectors. A phase consistency function is constructed based on the ratio of the sum of amplitudes to the sum of the moduli represented by the complex phases at each sampling point.

[0010] As a further aspect of the present invention: S4 specifically includes: Guided by cognitive uncertainty, a local neighborhood topology structure of pathological stable features in the original feature space is constructed. The local neighborhood topology structure is used to characterize the similarity constraint relationship between various pathological stable features. Based on the local neighborhood topology, an anchor constraint field is established in the shared high-dimensional semantic space. The anchor constraint field is used to maintain the consistency of the topological relationship of pathological stable features before and after mapping. By minimizing the distortion energy of the local neighborhood topology before and after mapping, the pathological stable features are iteratively projected to the corresponding positions in the shared high-dimensional semantic space, forming a multimodal joint feature representation with consistency in pathological evolution.

[0011] As a further aspect of the present invention: the establishment of the anchor point constraint field specifically includes: Extract pathologically stable features with minimal cognitive uncertainty from the local neighborhood topology as structural anchors, and obtain the local neighborhood distribution pattern of the structural anchors in the original feature space. In a shared high-dimensional semantic space, similarity preservation constraints are constructed based on the local neighborhood distribution patterns of structural anchor points. The similarity preservation constraint is extended globally in the shared high-dimensional semantic space to form an anchor constraint field covering all pathological stable features.

[0012] As a further aspect of the present invention: S5 specifically includes: The multimodal joint feature representations obtained from different time nodes are arranged in temporal order to form a joint feature evolution sequence. The joint feature evolution sequence is then mapped to a continuous time function space using function space projection to obtain a continuous evolution trajectory function. In the continuous time function space, evolution acceleration nodes are identified based on the local curvature changes of the evolution trajectory function, and the tangential offset of the evolution trajectory function at the evolution acceleration node is extracted. The tangential offset is input into a time-series prediction framework based on a probabilistic graphical structure. By calculating the probability amplitude of the current evolutionary trajectory deviating from the normal evolutionary path, the probability of early warning of cerebral infarction is output.

[0013] A multimodal fusion-based early warning system for cerebral infarction, comprising: The data acquisition module acquires multimodal data, including imaging data, serological data, and clinical scale data. The feature transformation module applies a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; The feature decoupling module, based on a heterogeneous feature set, performs heterogeneous feature decoupling in the function space using a sparse Gaussian process, respectively fitting the local structural stability and global volatility of each modality data, outputting the decoupled pathological stability features and physiological nonspecific noise features, and calculating the cognitive uncertainty corresponding to the pathological stability features. The feature reconstruction module dynamically reconstructs the pathological stable features based on cognitive uncertainty, maps the pathological stable features to a shared high-dimensional semantic space, and forms a multimodal joint feature representation with consistency in pathological evolution. The early warning output module inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework. By analyzing the evolution trend of the multimodal joint feature representation over time, it outputs the probability of ultra-early warning of cerebral infarction.

[0014] The beneficial effects of this invention are: (1) This invention decouples the heterogeneous feature set through a sparse Gaussian process and constructs a local energy spectral density using a phase consistency function. It calculates the cognitive uncertainty by combining the variance distribution parameters of the pathological stable features. This can effectively distinguish between the weak signal of the real infarct core area in imaging and the instantaneous physiological parameter fluctuations caused by factors such as tension and pain in clinical modality. It avoids the attention mechanism from incorrectly focusing the fusion weights on non-specific signals that are numerically significant but clinically insignificant, thereby reducing the false positive rate of ultra-early warning and improving the reliability of the warning results.

[0015] (2) This invention maps pathological stability features to a shared high-dimensional semantic space by constructing an anchor constraint field, forming a multimodal joint feature representation with pathological evolution consistency. It also identifies evolution acceleration nodes by combining evolution trajectory functions in continuous time function space, extracts tangential offsets, and calculates the probability amplitude of deviation from the normal evolution path through a probabilistic graph framework. This can capture the temporal co-evolution trend of multimodal data in the ultra-early stage of cerebral infarction and achieve accurate early warning before the pathological state forms obvious imaging signs. Attached Figure Description

[0016] The invention will now be further described with reference to the accompanying drawings.

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system block diagram of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, this invention is a multimodal fusion method for early warning of cerebral infarction, comprising the following steps: S1, Acquire multimodal data, which includes imaging data, serological data, and clinical scale data; S2, apply a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; S3, based on the heterogeneous feature set, uses a sparse Gaussian process to perform heterogeneous feature decoupling in the function space, fits the local structural stability and global volatility of each modality data respectively, outputs the decoupled pathological stability features and physiological nonspecific noise features, and calculates the cognitive uncertainty corresponding to the pathological stability features. S4. Based on cognitive uncertainty, the pathological stable features are dynamically reconstructed with weights, and the pathological stable features are mapped to a shared high-dimensional semantic space to form a multimodal joint feature representation with consistency in pathological evolution. S5 inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework, and outputs the probability of ultra-early warning of cerebral infarction by analyzing the evolution trend of the multimodal joint feature representation in time.

[0020] In S1, multimodal data is acquired, including imaging data, serological data, and clinical scale data, specifically including: The imaging data was acquired by using a magnetic resonance imaging (MRI) device to perform a cranial scan on the subject. The scan sequence included at least a diffusion-weighted imaging sequence and a liquid attenuation inversion recovery sequence. The acquisition parameters were set to a slice thickness of 5 mm and a slice spacing of 1.5 mm. The subject's cranial diffusion-weighted imaging images and liquid attenuation inversion recovery images were acquired to constitute the imaging data.

[0021] Serological data were obtained by collecting venous blood samples from the subjects, using a fully automated biochemical analyzer to detect the concentration of high-sensitivity C-reactive protein in the serum, and using a fully automated immunoassay analyzer to detect the concentrations of S100 calcium-binding protein B and matrix metalloproteinase 9 in the serum. The results of the above three tests were used as serological data.

[0022] The clinical scale data is obtained by presenting the National Institutes of Health Stroke Scale through a human-computer interaction interface. Clinicians fill in the scale scoring items one by one based on the results of the physical examination and medical history inquiry of the examinee, thereby generating clinical scale data.

[0023] In S2, a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping is applied to the multimodal data to generate heterogeneous feature sets embedded with spatial location basis and temporal evolution basis, specifically including: A joint transformation based on variational mode decomposition (MODED) and stochastic Fourier eigenmaps is applied to the imaging data to generate a spatial location basis. Specifically, the diffusion-weighted imaging images in the imaging data are first constructed into a three-dimensional topological graph structure. This 3D topological graph structure uses voxel points as nodes and the spatial neighborhood relationships between voxel points as edges, with the neighborhood radius set to 1.5 times the distance between the centers of two adjacent voxels. For this 3D topological graph structure, variational mode decomposition is used to decompose the signal intensity sequence of each voxel point. Variational mode decomposition is achieved by constructing a variational problem with the objective of minimizing the sum of the bandwidths of all intrinsic mode functions (IMFs). The constraint condition is that the sum of all IMFs equals the original signal intensity sequence. This variational problem is solved iteratively using the alternating direction multiplier method, decomposing the original signal intensity sequence into 3 to 5 IMFs. From the intrinsic mode functions obtained by decomposition, intrinsic mode components with center frequencies in the range of 0.02 Hz to 0.08 Hz are selected. This frequency range corresponds to the characteristic frequency band of tissue microstructure changes. The selected intrinsic mode components are used as spatial location basis.

[0024] A joint transformation based on variational mode decomposition and stochastic Fourier eigenmaps was applied to serological data to generate a time evolution basis. Specifically, the concentrations of high-sensitivity C-reactive protein, S100 calcium-binding protein B, and matrix metalloproteinase 9 in the serological data were first arranged in chronological order according to the sampling time points, constructing three independent time series. For each time series, variational mode decomposition was performed, constructed in the same manner as described above, using an iterative solution with alternating direction multipliers to decompose each time series into three intrinsic mode functions (EMFs). From the obtained EMFs, the lowest frequency EMF component, which characterizes the persistent evolution trend and has a center frequency below 0.01 Hz, was selected as the time evolution basis. The same variational mode decomposition process was applied to the clinical scale data. The scores of the National Institutes of Health Stroke Scale were arranged in chronological order according to the time points of collection. After constructing a time series, the lowest frequency intrinsic mode components were decomposed and extracted, which were also used as the time evolution basis.

[0025] The spatial location basis and temporal evolution basis are respectively mapped to a high-dimensional regenerating kernel Hilbert space using random Fourier feature mapping. The specific implementation of random Fourier feature mapping is as follows: For the spatial location basis, a Gaussian kernel function is constructed based on the spatial Euclidean distance between nodes in the 3D topological graph structure, with the width parameter of the Gaussian kernel function set to twice the median of the Euclidean distances between all nodes; for the temporal evolution basis, a Gaussian kernel function is constructed based on the time interval between each sampling point, with the width parameter of the Gaussian kernel function set to twice the average time interval between adjacent sampling points. 1000 Fourier basis functions are randomly sampled from their respective Gaussian kernel functions, and the spatial location basis and temporal evolution basis are projected onto the regenerating kernel Hilbert space spanned by these Fourier basis functions, respectively, to obtain the spatial location feature vector and temporal evolution feature vector in the high-dimensional space.

[0026] In the high-dimensional regenerative nuclear Hilbert space, the spatial location feature vector and the temporal evolution feature vector of the same subject are subjected to the Hadamard product operation. The Hadamard product operation refers to multiplying corresponding elements of the two feature vectors to generate a new feature vector. This new feature vector is the heterogeneous feature set that embeds the interaction between the spatial location basis and the temporal evolution basis. This heterogeneous feature set simultaneously contains information on tissue microstructural changes in imaging data and pathological evolution inertia information in serological data and clinical scale data.

[0027] In S3, based on the heterogeneous feature set, a sparse Gaussian process is used to perform heterogeneous feature decoupling in the function space. The local structural stability and global volatility of each modality are fitted separately, outputting the decoupled pathological stability features and physiological nonspecific noise features. The cognitive uncertainty corresponding to the pathological stability features is also calculated, specifically including: The heterogeneous feature set generated in step S2 is input into the sparse Gaussian process framework to perform heterogeneous feature decoupling in the function space. The sparse Gaussian process approximates the complete Gaussian process by selecting an induced point set. The number of induced points is set to one-tenth of the number of samples in the heterogeneous feature set, and the positions of the induced points are selected in the function space using a uniform sampling method. The sparse Gaussian process uses a covariance function to fit the local structural stability component and the global volatility component in the heterogeneous feature set, respectively. The covariance function is the squared exponential covariance function, and its length scale parameter is optimized by maximizing the marginal likelihood function. The optimization process uses the conjugate gradient method for iterative solution. After decoupling by the sparse Gaussian process, two feature sets are output: pathological stability features characterizing stable changes in tissue microstructure and physiological nonspecific noise features characterizing transient disturbance factors.

[0028] The decoupled pathological stability features and physiological nonspecific noise features are projected onto the regeneration kernel Hilbert space, respectively, and the variance distribution parameters of the pathological stability features in the regeneration kernel Hilbert space are extracted. Specifically, the projection method is as follows: using the same stochastic Fourier feature mapping method as in step S2, the pathological stability features are mapped to the regeneration kernel Hilbert space to obtain a high-dimensional pathological stability feature vector; then, the variance of each dimension of this high-dimensional pathological stability feature vector is calculated, and the arithmetic mean of the variances of all dimensions is used as the variance distribution parameter.

[0029] Based on the fluctuation amplitude of physiological nonspecific noise features in the function space, a local energy spectral density is constructed to characterize the degree of signal distortion. First, the instantaneous angular frequency of the physiological nonspecific noise features in the function space is extracted. Specifically, a Hilbert transform is performed on the trajectory of the physiological nonspecific noise features in the function space to obtain an analytic signal. The instantaneous angular frequency is obtained from the phase derivative of the analytic signal with respect to time. The fluctuation irregularity at each sampling point is determined based on the gradient change of the instantaneous angular frequency. The fluctuation irregularity is defined as the ratio of the standard deviation to the mean of the instantaneous angular frequency in the local neighborhood. The radius of the local neighborhood is set to three times the distance between adjacent sampling points.

[0030] Based on the fluctuation irregularity, a phase consistency function is constructed in the function space. Specifically, the instantaneous phase at each sampling point is determined according to the fluctuation irregularity in the function space. The instantaneous phase value at each sampling point is obtained by combining the original signal with its Hilbert transform result. The instantaneous phase is mapped onto the unit circle in the complex plane, i.e., the cosine and sine values ​​of the instantaneous phase at each sampling point are calculated, forming a complex representation. Within a local neighborhood of the function space, the complex representations are vector-superimposed, i.e., the real parts of all complex representations in the local neighborhood are added to obtain the total real part, and the imaginary parts are added to obtain the total imaginary part. The total real and imaginary parts constitute the superimposed vector. The resultant amplitude of the superimposed vector is obtained by taking the square root of the sum of the squares of the total real and imaginary parts, and the resultant phase angle is obtained by calculating the arctangent of the ratio of the total imaginary part to the total real part.

[0031] A phase coherence function is constructed based on the ratio of the combined amplitude to the sum of the complex moduli at each sampling point. The complex moduli at each sampling point are obtained by taking the square root of the sum of the squares of the real and imaginary parts. The sum of the moduli at all sampling points in the local neighborhood is then calculated. The formula for calculating the phase coherence function is as follows: ; in, This represents the phase coherence function value. This represents the resultant amplitude of the superimposed vectors. This represents the total number of sampling points within the local neighborhood. Represents the local neighborhood of the first The complex magnitude is represented at each sampling point. The phase coherence function value is between 0 and 1. When the instantaneous phases of each sampling point in a local neighborhood are highly consistent, the sum of the magnitudes is close to the sum of the magnitudes, and the phase coherence function value approaches 1, representing structural fluctuations. When the instantaneous phases of each sampling point are randomly distributed, the sum of the magnitudes is much smaller than the sum of the magnitudes, and the phase coherence function value approaches 0, representing random disturbances.

[0032] Within a local neighborhood of the phase consistency function space, energy is weighted and aggregated according to the phase consistency function to obtain the local energy spectral density reflecting the degree of signal distortion. Specifically: with each sampling point as the center, the radius of the local neighborhood is set to twice the distance between adjacent sampling points. The energy value of each sampling point is calculated within the neighborhood, and the energy value is defined as the square of the amplitude of the physiological nonspecific noise feature at that sampling point. The energy value of the sampling point is multiplied by the phase consistency function value at that sampling point to obtain the weighted energy value. Then, the weighted energy values ​​of all sampling points in the local neighborhood are summed to obtain the local energy spectral density at that sampling point.

[0033] The ratio between the variance distribution parameter of pathological stable features and the local energy spectral density of physiological nonspecific noise features is used as an uncertainty quantification factor. A nonlinear normalization transformation is performed on the ratio to output cognitive uncertainty. The nonlinear normalization transformation uses a logarithmic transformation, i.e., after calculating the natural logarithm of the ratio, it is compressed using a logical function to make the output value between 0 and 1. The formula for calculating cognitive uncertainty is: ; in, Indicates cognitive uncertainty, Variance distribution parameters representing pathological stability characteristics Local energy spectral density representing physiological nonspecific noise characteristics The adjustment coefficient is set to 0.5. When the variance distribution parameter of the pathological stable feature is relatively large and the local energy spectral density of the physiological nonspecific noise feature is relatively small, the cognitive uncertainty approaches 0, indicating that the algorithm has a high cognitive confidence in the pathological stable feature. Conversely, when the local energy spectral density of the physiological nonspecific noise feature is relatively large, the cognitive uncertainty approaches 1, indicating that the feature is susceptible to noise interference and its fusion weight should be reduced. The calculated cognitive uncertainty is used for dynamic weight reconstruction of the pathological stable feature in subsequent steps.

[0034] In S4, the pathological stability features are dynamically reconstructed based on cognitive uncertainty, mapping them to a shared high-dimensional semantic space to form a multimodal joint feature representation with consistency in pathological evolution. Specifically, this includes: Guided by the cognitive uncertainty calculated in step S3, a local neighborhood topology of the pathological stability features in the original feature space is constructed. Specifically, each pathological stability feature output in step S3 is considered as a sample point in the original feature space. The Euclidean distance between any two sample points is calculated by taking the square root of the sum of the squares of the differences in each dimension between the two sample points. For each sample point, the k sample points with the smallest Euclidean distance are selected as neighborhood points, and the value of k is set to one-tenth of the total number of sample points, rounded up. Using the sample points as nodes and the connections between sample points and neighborhood points as edges, a local neighborhood topology is constructed. The weight of each edge is set to the reciprocal of the product of the cognitive uncertainties between the two sample points. The smaller the cognitive uncertainty, the greater the weight of the connection between sample points, indicating that the connection has a higher similarity constraint relationship.

[0035] Based on the local neighborhood topology, an anchor point constraint field is established in the shared high-dimensional semantic space. First, the pathologically stable feature with the minimum cognitive uncertainty is extracted from the local neighborhood topology as the structural anchor point. Specifically, all pathologically stable features are traversed, and the feature with the smallest cognitive uncertainty is selected as the structural anchor point. The local neighborhood distribution pattern of this structural anchor point in the original feature space is obtained, i.e., the Euclidean distance between the structural anchor point and its neighbors, and the direction vectors of each neighbor point relative to the structural anchor point are obtained. The direction vectors are calculated by subtracting the coordinates of the neighbor points from the coordinates of the structural anchor point. In the shared high-dimensional semantic space, a similarity preservation constraint is constructed based on the local neighborhood distribution pattern of the structural anchor point. This constraint requires that after mapping to the shared high-dimensional semantic space, the distance ratio between each neighbor point and the structural anchor point remains consistent with the distance ratio in the original feature space. The similarity preservation constraint is extended globally in the shared high-dimensional semantic space. Specifically, starting from the structural anchor point, the similarity preservation constraint is propagated layer by layer to the sample points directly connected to the structural anchor point based on the connection relationship in the local neighborhood topology. Then, it is propagated from these sample points to the next layer of sample points until all pathological stable features are covered, forming an anchor point constraint field.

[0036] By minimizing the distortion energy of the local neighborhood topology before and after mapping, pathological stable features are iteratively projected to their corresponding positions in the shared high-dimensional semantic space. Distortion energy is defined as the square of the difference between the Euclidean distance between each pair of sample points in the original feature space and the Euclidean distance between corresponding sample points in the shared high-dimensional semantic space, multiplied by the connection weight between the pair of sample points, and the sum of these products for all sample point pairs yields the total distortion energy. The gradient descent method is used to iteratively optimize the coordinate positions of each pathological stable feature in the shared high-dimensional semantic space. In each iteration, the total distortion energy at the current coordinate position is calculated, and the gradient of the total distortion energy with respect to the coordinates of each sample point is calculated. The coordinate positions are updated along the negative gradient direction with a step size of 0.01. Iteration continues until the rate of change of the total distortion energy between two adjacent iterations is less than one-thousandth. At this point, the coordinate positions of each pathological stable feature in the shared high-dimensional semantic space are the final projection results. By combining the coordinate positions of all pathological stable features in a shared high-dimensional semantic space, a multimodal joint feature representation with pathological evolution consistency is formed. The relative positional relationship of pathological stable features of different modalities in the semantic space reflects their intrinsic correlation in the pathological evolution process.

[0037] In S5, multimodal joint feature representations are input into a pre-constructed Bayesian temporal prediction framework. By analyzing the temporal evolution trend of the multimodal joint feature representations, the probability of ultra-early warning of cerebral infarction is output, specifically including: The multimodal joint feature representations acquired at different time points are arranged in chronological order to form a joint feature evolution sequence. Specifically, for the same subject, the multimodal joint feature representations acquired at multiple time points and processed through steps S1 to S4 are arranged sequentially according to the acquisition time. Each time point corresponds to a multimodal joint feature representation vector, and the representation vectors of all time points together constitute the joint feature evolution sequence. The time interval between two adjacent time points in this sequence is set according to the actual acquisition settings, ranging from 6 hours to 24 hours. The joint feature evolution sequence is mapped to a continuous time function space using function space projection to obtain a continuous evolution trajectory function. Specifically, the acquisition time point is used as the independent variable, and the components of each dimension of the multimodal joint feature representation vector corresponding to each time point are used as the dependent variable. A continuous evolution trajectory function is constructed for each dimension using cubic spline interpolation. The specific calculation process of cubic spline interpolation is as follows: a cubic polynomial function is constructed between two adjacent acquisition time points. The function value of the cubic polynomial function is equal to that of the original data point at the endpoints, and its first and second derivatives are continuous at the internal nodes. By solving the trilinear equation system composed of these continuity conditions, the coefficients of the cubic polynomial function in each interval are obtained, thereby obtaining the evolution trajectory function that is continuously differentiable over the entire time interval.

[0038] In a continuous-time function space, evolutionary acceleration nodes are identified based on the local curvature changes of the evolutionary trajectory function. The local curvature of the evolutionary trajectory function at any time point is calculated as follows: the first and second derivative vectors of the evolutionary trajectory function at that time point are calculated. The first derivative vector represents the evolutionary velocity, and the second derivative vector represents the evolutionary acceleration. The local curvature is equal to the magnitude of the cross product of the first and second derivative vectors divided by the cube of the magnitude of the first derivative vector. By traversing the entire continuous-time function space, the local curvature value at each time point is calculated. Time points whose local curvature values ​​are greater than 1.5 times the average local curvature of all sampling points are marked as evolutionary acceleration nodes. The tangential offset of the evolutionary trajectory function at the evolution acceleration node is extracted. The tangential offset is calculated as follows: taking the evolution acceleration node as the starting point, extend a preset duration window along the positive time axis. This duration window is set to twice the interval between adjacent acquisition time points. Calculate the difference vector between the endpoint coordinates and the starting point coordinates of the evolutionary trajectory function within this duration window. Then project this difference vector onto the tangential direction of the evolutionary trajectory function at the starting point. The scalar value obtained by projection is the tangential offset. This tangential offset is used to quantify the magnitude of the evolutionary path deviating from the original direction.

[0039] The tangential offset is input into a probabilistic graphical temporal prediction framework. By calculating the probability amplitude of the current evolutionary trajectory deviating from the normal evolutionary path, the probability of ultra-early warning of cerebral infarction is output. The construction method of the probabilistic graphical temporal prediction framework is as follows: First, multimodal joint feature representations of healthy individuals within the same time span are collected. The joint feature evolution sequence of healthy individuals is constructed in the same way as described above, and the local curvature and tangential offset at each time point are calculated to obtain the reference distribution of tangential offset of the normal evolutionary path. This temporal prediction framework adopts a directed acyclic graph structure. Nodes in the graph correspond to different time points, and directed edges between nodes represent temporal transition relationships. A state variable is set at each node, and the state variable takes the value of normal state or abnormal state. For the current subject, the tangential offset extracted at the evolution acceleration node is compared with the reference distribution of tangential offset of the normal evolutionary path, and the cumulative probability value of the current tangential offset in the reference distribution is calculated. This cumulative probability value represents the probability of the current tangential offset occurring. The cumulative probability value is used as the prior probability of the abnormal state and propagated along the directed acyclic graph structure. Specifically, starting from the first time point, the probability of being in an abnormal state at each time point is calculated sequentially. This probability equals the probability of being in an abnormal state at the previous time point multiplied by the state transition probability, plus the observed prior probability of the abnormal state at the current time point, where the state transition probability is set to 0.1. After probability propagation through all time points, the probability value of being in an abnormal state at the last time point is output as the early warning probability of cerebral infarction. This warning probability is between 0 and 1. When the warning probability is greater than 0.6, it indicates that the subject is in a high-risk state of early cerebral infarction.

[0040] Please see Figure 2 As shown, a multimodal fusion early warning system for cerebral infarction includes: The data acquisition module acquires multimodal data, including imaging data, serological data, and clinical scale data. The feature transformation module applies a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; The feature decoupling module, based on a heterogeneous feature set, performs heterogeneous feature decoupling in the function space using a sparse Gaussian process, respectively fitting the local structural stability and global volatility of each modality data, outputting the decoupled pathological stability features and physiological nonspecific noise features, and calculating the cognitive uncertainty corresponding to the pathological stability features. The feature reconstruction module dynamically reconstructs the pathological stable features based on cognitive uncertainty, maps the pathological stable features to a shared high-dimensional semantic space, and forms a multimodal joint feature representation with consistency in pathological evolution. The early warning output module inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework. By analyzing the evolution trend of the multimodal joint feature representation over time, it outputs the probability of ultra-early warning of cerebral infarction.

[0041] The working principle of this invention is as follows: First, multimodal data, including imaging data, serological data, and clinical scale data of the subjects, are acquired. Then, spatial location basis representing changes in tissue microstructure is extracted from the imaging data using a three-dimensional topological graph structure combined with variational mode decomposition. Temporal evolution basis representing the pathological evolution inertia is extracted from the serological and clinical scale data using variational mode decomposition. The spatial location basis and temporal evolution basis are then projected onto a high-dimensional regenerative kernel Hilbert space using stochastic Fourier feature mapping, followed by Hadamard product operation to generate a heterogeneous feature set embedding the interaction between the spatial location basis and the temporal evolution basis. Next, a sparse Gaussian process is used to decouple the heterogeneous feature set in the function space, outputting pathological stability features and physiological nonspecific noise features. Local energy spectral density is calculated by constructing a phase consistency function, combined with the variance of the pathological stability features. The cognitive uncertainty is calculated using parameters. Then, guided by the cognitive uncertainty, a local neighborhood topology of the pathological stable features in the original feature space is constructed. Features with the minimum cognitive uncertainty are selected as structural anchors to establish an anchor constraint field. By minimizing the distortion energy of the topology before and after mapping, the pathological stable features are iteratively projected into a shared high-dimensional semantic space to form a multimodal joint feature representation with pathological evolution consistency. Finally, the multimodal joint feature representations of different time nodes are arranged into a joint feature evolution sequence. A continuous evolution trajectory function is obtained through cubic spline interpolation. Evolution acceleration nodes are identified based on local curvature changes, and tangential offsets are extracted. The tangential offsets are input into a probabilistic graph framework based on a directed acyclic graph structure. The probability amplitude of the current evolution trajectory deviating from the normal evolution path is calculated through probability propagation, and the probability of ultra-early warning of cerebral infarction is output.

[0042] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A multimodal fusion method for early warning of cerebral infarction, characterized in that, Includes the following steps: S1, Acquire multimodal data, which includes imaging data, serological data, and clinical scale data; S2, apply a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data respectively to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; S3, based on the heterogeneous feature set, uses a sparse Gaussian process to perform heterogeneous feature decoupling in the function space, fits the local structural stability and global volatility of each modality data respectively, outputs the decoupled pathological stability features and physiological nonspecific noise features, and calculates the cognitive uncertainty corresponding to the pathological stability features. S4. Based on cognitive uncertainty, the pathological stable features are dynamically reconstructed with weights, and the pathological stable features are mapped to a shared high-dimensional semantic space to form a multimodal joint feature representation with consistency in pathological evolution. S5 inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework, and outputs the probability of ultra-early warning of cerebral infarction by analyzing the evolution trend of the multimodal joint feature representation in time.

2. The multimodal fusion method for early warning of cerebral infarction according to claim 1, characterized in that, S2 specifically includes: A three-dimensional topological graph structure is constructed based on the spatial neighborhood relationship of imaging data. Variational mode decomposition is used to decompose the voxel intensity sequence in the three-dimensional topological graph structure into several intrinsic mode functions. The intrinsic mode components corresponding to the changes in tissue microstructure are extracted as spatial location basis. For serological data and clinical scale data, time series were constructed respectively. The transient fluctuations and continuous evolution trends in the time series were separated by variational mode decomposition, and the intrinsic mode components that characterize the inertia of pathological evolution were extracted as the time evolution basis. The spatial location basis and the temporal evolution basis are respectively mapped to a high-dimensional regenerating kernel Hilbert space through random Fourier feature mapping. The two types of basis are then subjected to Hadamard product operation in the high-dimensional regenerating kernel Hilbert space to generate a heterogeneous feature set that embeds the interaction relationship between the spatial location basis and the temporal evolution basis.

3. The multimodal fusion method for early warning of cerebral infarction according to claim 1, characterized in that, S3 specifically includes: The decoupled pathological stability features and physiological nonspecific noise features are projected onto the regeneration kernel Hilbert space respectively, and the variance distribution parameters of the pathological stability features in the regeneration kernel Hilbert space are extracted. Based on the fluctuation amplitude of physiological nonspecific noise characteristics in the function space, a local energy spectral density is constructed to characterize the degree of signal distortion. The ratio between the variance distribution parameter of pathological stable features and the local energy spectral density of physiological nonspecific noise features is used as an uncertainty quantification factor. The cognitive uncertainty is output by performing a nonlinear normalization transformation through the comparison value.

4. The multimodal fusion method for early warning of cerebral infarction according to claim 3, characterized in that, The construction of the local energy spectral density used to characterize the degree of signal distortion specifically includes: Extract the instantaneous angular frequency of physiological nonspecific noise features in the function space, and determine the fluctuation irregularity at each sampling point based on the gradient change of the instantaneous angular frequency; Based on the volatility irregularity, a phase consistency function is constructed in the function space. The phase consistency function is used to distinguish between structural volatility and random disturbance. Within a local neighborhood of the phase coherence function space, energy is weighted and aggregated according to the phase coherence function to obtain a local energy spectral density that reflects the degree of signal distortion.

5. The multimodal fusion method for early warning of cerebral infarction according to claim 4, characterized in that, The construction of the phase consistency function specifically includes: Based on the fluctuation irregularity, the instantaneous phase at each sampling point in the function space is determined, and the instantaneous phase is mapped onto the unit circle of the complex plane to obtain the complex phase characterization; Within a local neighborhood of the function space, the complex phase representations are superimposed to obtain the resultant amplitude and resultant phase angle of the superimposed vectors. A phase consistency function is constructed based on the ratio of the sum of amplitudes to the sum of the moduli represented by the complex phases at each sampling point.

6. The multimodal fusion method for early warning of cerebral infarction according to claim 1, characterized in that, S4 specifically includes: Guided by cognitive uncertainty, a local neighborhood topology structure of pathological stable features in the original feature space is constructed. The local neighborhood topology structure is used to characterize the similarity constraint relationship between various pathological stable features. Based on the local neighborhood topology, an anchor constraint field is established in the shared high-dimensional semantic space. The anchor constraint field is used to maintain the consistency of the topological relationship of pathological stable features before and after mapping. By minimizing the distortion energy of the local neighborhood topology before and after mapping, the pathological stable features are iteratively projected to the corresponding positions in the shared high-dimensional semantic space, forming a multimodal joint feature representation with consistency in pathological evolution.

7. The multimodal fusion method for early warning of cerebral infarction according to claim 6, characterized in that, The establishment of the anchor point constraint field specifically includes: Extract pathologically stable features with minimal cognitive uncertainty from the local neighborhood topology as structural anchors, and obtain the local neighborhood distribution pattern of the structural anchors in the original feature space. In a shared high-dimensional semantic space, similarity preservation constraints are constructed based on the local neighborhood distribution patterns of structural anchor points. The similarity preservation constraint is extended globally in the shared high-dimensional semantic space to form an anchor constraint field covering all pathological stable features.

8. The multimodal fusion method for early warning of cerebral infarction according to claim 1, characterized in that, S5 specifically includes: The multimodal joint feature representations obtained from different time nodes are arranged in temporal order to form a joint feature evolution sequence. The joint feature evolution sequence is then mapped to a continuous time function space using function space projection to obtain a continuous evolution trajectory function. In the continuous time function space, evolution acceleration nodes are identified based on the local curvature changes of the evolution trajectory function, and the tangential offset of the evolution trajectory function at the evolution acceleration node is extracted. The tangential offset is input into a time-series prediction framework based on a probabilistic graphical structure. By calculating the probability amplitude of the current evolutionary trajectory deviating from the normal evolutionary path, the probability of early warning of cerebral infarction is output.

9. A multimodal fusion early warning system for cerebral infarction, characterized in that, A method for performing a multimodal fusion-based early warning system for cerebral infarction as described in any one of claims 1-8, comprising: The data acquisition module acquires multimodal data, including imaging data, serological data, and clinical scale data. The feature transformation module applies a joint transformation based on variational mode decomposition and stochastic Fourier feature mapping to the multimodal data to generate a heterogeneous feature set embedded with spatial location basis and temporal evolution basis; The feature decoupling module, based on a heterogeneous feature set, performs heterogeneous feature decoupling in the function space using a sparse Gaussian process, respectively fitting the local structural stability and global volatility of each modality data, outputting the decoupled pathological stability features and physiological nonspecific noise features, and calculating the cognitive uncertainty corresponding to the pathological stability features. The feature reconstruction module dynamically reconstructs the pathological stable features based on cognitive uncertainty, maps the pathological stable features to a shared high-dimensional semantic space, and forms a multimodal joint feature representation with consistency in pathological evolution. The early warning output module inputs the multimodal joint feature representation into a pre-constructed Bayesian temporal prediction framework. By analyzing the evolution trend of the multimodal joint feature representation over time, it outputs the probability of ultra-early warning of cerebral infarction.

Images