Cognitive disorder non-invasive screening system and method based on multi-modal characteristics

Abstract

The invention relates to the technical field of cognitive impairment noninvasive screening, and discloses a cognitive impairment noninvasive screening system and method.The system comprises a physiological sensing end, a feature collaboration end, an intelligent decision-making end and a visual report end, by arranging the physiological sensing end, 5-meter-distance non-contact detection is achieved, the feature collaboration end is connected with the intelligent decision-making end, and the visual report end is connected with the feature collaboration end; discomfort and signal interference caused by a traditional contact type electrode are avoided; the problem of asynchronous data fusion is solved by setting a feature collaboration end; according to the method, the dependence of traditional screening on contact equipment and professionals is broken through, high-precision and low-cost cognitive impairment general screening is realized, and the method is suitable for community medical treatment and family health monitoring scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of non-invasive screening for cognitive impairment, and specifically to a non-invasive screening system and method for cognitive impairment based on multimodal features. Background Art

[0002] The intelligent screening system for cognitive impairment is a medical detection technology that integrates multimodal behavioral analysis. Its core technology uses non-contact sensors to simultaneously collect eye movement, voice, and gait data to establish an early warning model for neurodegenerative diseases. The system currently uses an infrared eye tracker, a far-field microphone array, and a pressure track sensor as key data acquisition components, aiming to achieve high-precision behavioral feature extraction in natural conditions.

[0003] However, existing technologies have three major technical bottlenecks: limited screening methods: traditional screening relies on neuropsychological scales, which require face-to-face operation by professional physicians for more than 30 minutes, and the efficiency of community screening is low. Clinical data show that the sensitivity of the scale method to mild cognitive impairment is only 68%, and it is affected by language and cultural background; contact devices need to be attached to the human body, changing natural behavior patterns. Studies have confirmed that wearing an EEG cap causes elderly subjects to have a slower gait speed and an abnormally increased eye movement scanning frequency, resulting in behavioral data distortion; single-modality detection blind spots: single-modality technology cannot capture the multi-dimensional characteristics of cognitive impairment. For example, eye trackers can identify visual attention deficits, but Unable to detect speech and semantic degradation. Gait analyzers can detect abnormal motor coordination, but ignore micro-expression clues of decreased executive function. Independent modality screening has a high misdiagnosis rate and is insufficient to identify subtypes such as Lewy body dementia. Multimodal collaboration bottleneck: Existing multimodal systems cause data asynchrony due to differences in sampling frequency. Harvard Medical School experiments have shown that uncorrected asynchronous data increases the error rate of cross-modal correlation analysis. The problem of physiological rhythm imbalance has long been unsolved: There is a natural coupling between the eye movement saccadic rhythm and the gait frequency in healthy people, but the rhythm separation of patients with cognitive impairment is as high as 0.35. Existing technologies lack a quantitative correction mechanism and cannot identify such key pathological features. Therefore, a non-invasive screening system for cognitive impairment based on dynamic rhythm coordination and adaptive decision-making is proposed to solve the above problems. Summary of the Invention

[0004] (1) Technical problems solved In response to the deficiencies of the existing technology, the present invention provides a non-invasive screening system and method for cognitive impairment based on multimodal features, which solves the problems raised in the above background technology.

[0005] (2) Technical solution To achieve the above objectives, the present invention provides the following technical solutions: a non-invasive screening system and method for cognitive impairment based on multimodal features, the method comprising the following steps: S1. Use non-contact sensing equipment to synchronously collect multimodal physiological behavior data of the subject, including eye movement trajectory data, voice interaction data, and gait dynamic data; S2. Extracting spatiotemporal features from the eye movement trajectory data, voice interaction data, and gait dynamic data to generate an eye movement feature matrix, a voice feature matrix, and a gait feature matrix; S3. Aligning the temporal dimensions of the eye movement feature matrix, the speech feature matrix, and the gait feature matrix based on a dynamic time warping algorithm to construct a multimodal spatiotemporal fusion feature tensor; S4, calculating the physiological rhythm consistency index of the eye movement trajectory data and the gait dynamic data, and executing S6 when the rhythm consistency threshold meets the preset condition, otherwise executing S5; S5. Using a phase synchronization algorithm to perform rhythm resampling on the gait dynamic data, generating rhythm-calibrated gait data and updating the multimodal spatiotemporal fusion feature tensor; S6. Inputting the multimodal spatiotemporal fusion feature tensor into a pre-constructed cognitive impairment screening knowledge graph, wherein the knowledge graph comprises clinical diagnosis rule nodes, neuroimaging feature nodes, and behavioral pattern association edges; S7. Traverse the knowledge graph based on a reinforcement learning decision tree, match the optimal screening strategy and generate a screening decision path, wherein the decision path includes a feature weight allocation scheme; S8. Perform weighted fusion on the multimodal spatiotemporal fusion feature tensor according to the feature weight distribution scheme, and output a cognitive impairment risk index and a multidimensional behavior degradation heat map.

[0006] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S1 specifically includes: S11. Use an infrared eye tracker to collect eye movement data during the visual exploration task at a sampling rate greater than 120 Hz, and record pupil position change vectors. S12. Collect voice command response data through a directional microphone array and extract fundamental frequency disturbance parameters and semantic coherence scores; S13. Gait sequences were collected using a pressure-sensing walkway at a sampling rate of 100 Hz to generate a spatiotemporal matrix of plantar pressure distribution.

[0007] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S2 comprises: S21. Use a convolutional recurrent neural network to extract the saccade acceleration features and gaze point clustering features from the eye movement trajectory data to generate an 8-dimensional eye movement feature vector; S22. Extract acoustic fingerprint features from voice interaction data through Mel-frequency cepstral coefficient decomposition, and use a bidirectional LSTM network to generate a 16-dimensional voice time series feature matrix; S23. Wavelet packet decomposition is used to extract the trajectory characteristics of the pressure center of the support phase from the gait dynamic data, and a 12-dimensional gait dynamic parameter matrix is ​​constructed.

[0008] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S3 comprises: S31. Construct a three-dimensional feature tensor space, whose dimensions are defined as: time axis T, feature axis F, and modal axis M; S32, using a dynamic time warping algorithm to calculate the time offset δt of the eye movement-speech feature sequence, and unifying the three modal data to the same time baseline through an interpolation algorithm; S33. Stack the aligned feature matrices according to the modal dimension to generate a multimodal spatiotemporal fusion feature tensor with a dimension of [T×36×3].

[0009] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S4 comprises: S41. Calculate the power spectrum density of the saccade rhythm of the eye movement trajectory and the cadence power spectral density of gait data ; S42. Define rhythm consistency indicators: ; in, is an index of rhythm consistency. is the power spectral density function, is the rhythmic characteristic of the eye movement trajectory, is the rhythmic characteristic of gait dynamics, is the frequency differential element, when When ≤0.15, it was judged as consistent rhythm; S43, when When the value is >0.15, the rhythm resampling process is triggered; otherwise, feature tensor fusion is performed directly.

[0010] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S5 comprises: S51. Constructing a gait rhythm phase space model: ; in, is the instantaneous phase of the gait rhythm, for It's time function, is a mathematical constant, is the instantaneous frequency function of gait, is the time variable, is the time element; S52, use phase-locked loop technology to reduce the eye movement scanning frequency As a reference signal, adjust the gait phase to the synchronous state; S53. Resample the gait data at the synchronous phase points using cubic spline interpolation.

[0011] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S6 comprises: S61. The knowledge graph comprises a three-layer structure: The top layer is the clinical diagnosis rule node, which includes the MMSE score threshold and CDR grading standard; The middle layer is the neuroimaging feature node, which includes hippocampal atrophy rate and default network connection strength; The bottom layer is a multimodal behavior feature node, which is directly associated with the multimodal spatiotemporal fusion feature tensor; S62. Each node is connected by Bayesian probability edges, and the edge weights are trained by historical diagnostic data.

[0012] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S7 includes: S71. Construct the state space S = {knowledge graph node, current feature weight} of the reinforcement learning decision tree; S72. Define action space A = {node jump direction, weight adjustment range}; S73. Set reward function: ; in, is the reinforcement learning reward value, is the diagnostic accuracy weight factor, To verify the accuracy of clinical is the feature coverage weight factor, is the effective feature utilization, is the path length penalty factor, is the decision path complexity; S74. Iteratively optimize the decision path through the Q-learning algorithm until convergence.

[0013] Preferably, according to the non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, S8 comprises: S81. Calculate the weighted feature vector according to the weight distribution scheme output by the decision path: ; in, is the weighted eigenvector, is the modal weight coefficient, is the normalization operator, is the normalized modal feature; S82, will Input the radial basis function network and output the cognitive impairment risk index in the range of 0-1; S83. Generate a three-dimensional heat map based on feature contribution, marking abnormal areas of eye movement scanning paths, speech rhythm breakpoints, and gait symmetry defect areas.

[0014] Preferably, a real-time calibration system for cognitive impairment screening based on the method according to any one of claims 1 to 9 comprises the following steps: S91, real-time collection of environmental interference parameters: Obtain the ambient light intensity value L through the light sensor; Obtain the background noise decibel value dB through a sound pressure meter; Obtain the ground vibration amplitude A through the accelerometer; S92. Calculate the multimodal compensation coefficient: ; in, is the multimodal compensation coefficient, is the hyperbolic tangent function, is the ambient light intensity, is the background noise decibel value, is the ground vibration amplitude, is the illumination compensation weight, is the light intensity scaling factor, the second term of the formula is the noise compensation weight, is the noise baseline value, the third term of the formula is the vibration compensation weight, is the vibration amplitude benchmark; S92. Calculate the multimodal compensation coefficient: ; in, is the illumination component weight coefficient, is the hyperbolic tangent function, is the light intensity scaling factor, the second term in the formula is the noise component weight coefficient, the third term of the formula is the vibration component weight coefficient, is the ground vibration amplitude, when > 1.0 triggers compensation mode; S93, performing multimodal data compensation: Eye movement data compensation: using window size:

[0015] in, is the multimodal compensation coefficient, As the basic frame rate benchmark; Voice data compensation: Apply cutoff frequency: ; in, is the system transfer function, for Transform variables, As the upper limit benchmark of the human voice frequency band, is the multimodal compensation coefficient; Gait data compensation: ; in, is the system transfer function, for Transform variables, is the pole location parameter; S94. Update the rhythm consistency determination threshold: ; in, is an index of rhythm consistency. is the abbreviation of threshold, is the multimodal compensation coefficient; S95, Reconstruction of anti-interference heat map: Overlaying the environmental interference distribution map on the 3D behavior degradation heat map; Define the interference mask: ; in, is the environmental interference mask, is the spatial horizontal coordinate, is the spatial ordinate, is the time frame number, is the behavioral abnormality gradient, is the environmental disturbance gradient; Output calibration heatmap: ; in, For heat map, To represent the heat map after calibration, To represent the original heat map, is the environmental interference mask matrix; S96. Execute the equipment self-test protocol: When 5 times in a row Value> hour: Reset the eye tracker reference coordinates to (0,0); Recalibrate the beamforming parameters of the microphone array; Update the pressure sensitivity coefficient of the trail sensor to ,in is the sensitivity, For pressure, is the sensitivity enhancement factor, is the initial calibration sensitivity.

[0016] (3) Beneficial effects Compared with the existing technology, the present invention provides a non-invasive screening system and method for cognitive impairment based on multimodal features, which has the following beneficial effects: 1. In this invention, by setting up physiological sensing terminals, during non-invasive screening for cognitive impairment, an embedded eye tracker array, a far-field voice unit, and an intelligent walkway sensor grid are used to synchronously collect eye movement trajectories, voice interaction, and gait dynamic data, achieving non-contact detection at a distance of 5 meters, avoiding the discomfort and signal interference caused by traditional contact electrodes. Infrared sampling ensures millimeter-level accuracy of pupil movement trajectories, and pressure-sensing walkways capture subtle changes in plantar pressure, enabling the system to eliminate environmental noise interference, solve the problem of behavioral data distortion caused by wearable devices, and ensure the authenticity and reliability of multimodal behavioral feature extraction.

[0017] 2. In the present invention, by setting a feature coordination terminal, when performing multimodal data analysis, a dynamic time warping algorithm is used to align the temporal dimensions of eye movement, voice, and gait, and a three-dimensional spatiotemporal fusion feature tensor is constructed to solve the problem of asynchronous data fusion; the eye movement-gait physiological rhythm consistency index is calculated through the rhythm synchronization coprocessor, and when the rhythm is abnormal, the phase synchronization algorithm is triggered to resample the gait data, so that the system can reduce the cross-modal feature alignment error, break through the technical bottleneck of the high missed diagnosis rate of traditional single-modal screening, and ensure that the sensitivity of early identification of cognitive impairment is improved.

[0018] 3. In the present invention, by setting up an intelligent decision-making terminal, when generating a screening strategy, an adaptive weight distribution scheme is generated using a reinforcement learning decision tree based on the clinical diagnosis rule nodes, neuroimaging feature nodes and behavioral pattern association edges of the knowledge graph; the decision path is dynamically optimized through the reward function, so that the system can complete accurate screening. At the same time, the generated three-dimensional behavioral degradation heat map can locate abnormal areas of eye movement scanning paths, speech rhythm breakpoints, and gait symmetry defect areas, thereby realizing the visual tracing of the causes of cognitive impairment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the method steps of the present invention; Figure 2 It is a schematic diagram of the overall system architecture of the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] See also Figure 1-2 The non-invasive screening system and method for cognitive impairment based on multimodal features comprises the following steps: S1. Use non-contact sensing equipment to synchronously collect multimodal physiological behavior data of the subject, including eye movement trajectory data, voice interaction data, and gait dynamic data; S2. Extract spatiotemporal features from the eye movement trajectory data, voice interaction data, and gait dynamic data to generate an eye movement feature matrix, a voice feature matrix, and a gait feature matrix; S3. Align the temporal dimensions of the eye movement feature matrix, speech feature matrix, and gait feature matrix based on the dynamic time warping algorithm to construct a multimodal spatiotemporal fusion feature tensor. S4, calculating the physiological rhythm consistency index of the eye movement trajectory data and the gait dynamic data, and executing S6 when the rhythm consistency threshold meets the preset condition, otherwise executing S5; S5. Use a phase synchronization algorithm to perform rhythm resampling on the gait dynamic data, generate rhythm-calibrated gait data, and update the multimodal spatiotemporal fusion feature tensor; S6. Input the multimodal spatiotemporal fusion feature tensor into a pre-built cognitive impairment screening knowledge graph, which contains clinical diagnosis rule nodes, neuroimaging feature nodes, and behavioral pattern association edges; S7. Traverse the knowledge graph based on a reinforcement learning decision tree, match the optimal screening strategy and generate a screening decision path, wherein the decision path includes a feature weight allocation scheme; S8. Performing weighted fusion on the multimodal spatiotemporal fusion feature tensors according to a feature weight distribution scheme, and outputting a cognitive impairment risk index and a multidimensional behavior degradation heat map; S11. Use an infrared eye tracker to collect eye movement data during the visual exploration task at a sampling rate greater than 120 Hz, and record pupil position change vectors. S12. Collect voice command response data through a directional microphone array and extract fundamental frequency disturbance parameters and semantic coherence scores; S13, using a pressure-sensing walkway to collect gait sequences at a sampling rate of 100 Hz to generate a spatiotemporal matrix of plantar pressure distribution; S21. Use a convolutional recurrent neural network to extract the saccade acceleration features and gaze point clustering features from the eye movement trajectory data to generate an 8-dimensional eye movement feature vector; S22. Extract acoustic fingerprint features from voice interaction data through Mel-frequency cepstral coefficient decomposition, and use a bidirectional LSTM network to generate a 16-dimensional voice time series feature matrix; S23, using wavelet packet decomposition to extract the pressure center trajectory characteristics of the stance phase from the gait dynamic data, and constructing a 12-dimensional gait dynamics parameter matrix; S31. Construct a three-dimensional feature tensor space, whose dimensions are defined as: time axis T, feature axis F, and modal axis M; S32, using a dynamic time warping algorithm to calculate the time offset δt of the eye movement-speech feature sequence, and unifying the three modal data to the same time baseline through an interpolation algorithm; S33, stacking the aligned feature matrices according to the modality dimension to generate a multimodal spatiotemporal fusion feature tensor with a dimension of [T×36×3]; S41. Calculate the power spectrum density of the saccade rhythm of the eye movement trajectory and the cadence power spectral density of gait data ; S42. Define rhythm consistency indicators: ; in, is an index of rhythm consistency. is the power spectral density function, is the rhythmic characteristic of the eye movement trajectory, is the rhythmic characteristic of gait dynamics, is the frequency differential element, when When ≤0.15, it was judged as consistent rhythm; S43, when When >0.15, the rhythm resampling process is triggered, otherwise the feature tensor fusion is directly executed; S51. Constructing a gait rhythm phase space model: ; in, is the instantaneous phase of the gait rhythm, for It's time function, is a mathematical constant, is the instantaneous frequency function of gait, is the time variable, is the time element; S52, use phase-locked loop technology to reduce the eye movement scanning frequency As a reference signal, adjust the gait phase to the synchronous state; S53, resampling the gait data at the synchronized phase points using cubic spline interpolation; S61. The knowledge graph consists of three layers: The top layer is the clinical diagnosis rule node, which includes the MMSE score threshold and CDR grading standard; The middle layer is the neuroimaging feature node, which includes hippocampal atrophy rate and default network connection strength; The bottom layer is the multimodal behavior feature node, which is directly associated with the multimodal spatiotemporal fusion feature tensor; S62, each node is connected by Bayesian probabilistic edges, and the edge weights are trained by historical diagnostic data; S71. Construct the state space S = {knowledge graph node, current feature weight} of the reinforcement learning decision tree; S72. Define action space A = {node jump direction, weight adjustment range}; S73. Set reward function:

[0022] in, is the reinforcement learning reward value, is the diagnostic accuracy weight factor, To verify the accuracy of clinical is the feature coverage weight factor, is the effective feature utilization, is the path length penalty factor, is the decision path complexity; S74. Iteratively optimize the decision path through the Q-learning algorithm until convergence; S81. Calculate the weighted feature vector according to the weight distribution scheme output by the decision path: ; in, is the weighted eigenvector, is the modal weight coefficient, is the normalization operator, is the normalized modal feature; S82, will Input the radial basis function network and output the cognitive impairment risk index in the range of 0-1; S83. Generate a three-dimensional heat map based on the feature contribution, marking the abnormal eye movement scanning path area, speech rhythm breakpoints and gait symmetry defect areas; S81. Calculate the weighted feature vector according to the weight distribution scheme output by the decision path: ; in, is the weighted eigenvector, is the modal weight coefficient, is the normalization operator, is the normalized modal feature; S82, will Input the radial basis function network and output the cognitive impairment risk index in the range of 0-1; S83. Generate a three-dimensional heat map based on feature contribution, marking abnormal areas of eye movement scanning paths, speech rhythm breakpoints, and gait symmetry defect areas. Specific embodiments Example 1: Multimodal Data Collection and Feature Fusion The embedded eye tracker array is used to capture the pupil movement trajectory of the subjects during the visual exploration task, and the 8-dimensional scanning acceleration characteristics are synchronously recorded. The far-field speech unit collects voice command response data, and the 16-dimensional acoustic fingerprint features are extracted through Mel-frequency cepstral coefficient decomposition. The intelligent trail sensor grid obtains the plantar pressure center trajectory and generates 12-dimensional gait dynamic parameters. The dynamic time warping algorithm is used to align the three modal time series dimensions: based on the starting point of the gait support phase, the time offset of the eye movement and speech data is interpolated and corrected to construct a 36×3-dimensional spatiotemporal fusion feature tensor.

[0024] Implementation effect: Non-contact detection at a distance of 5 meters is achieved in community screening, the environmental noise suppression rate is greater than 85%, and the cross-modal feature alignment error is less than 0.05 seconds.

[0025] Example 2: Circadian Rhythm Synchronization and Decision Optimization The absolute integral difference (CI) between the power spectrum of the eye movement rhythm and the gait frequency power spectrum was calculated. When the CI reached 0.28, the phase synchronization algorithm was triggered. Using the eye movement rhythm as a reference signal, the gait phase angle was adjusted using a phase-locked loop (PLL) technique. Rhythm-calibrated gait data was resampled and the updated feature tensor was input into the knowledge graph. Based on clinical rule nodes, neuroimaging nodes, and behavioral association edges, a reinforcement learning decision tree generated the optimal weighting scheme after 200 iterations.

[0026] Implementation effect: Rhythm synchronization increased the MCI identification sensitivity to 91.4%, and decision path optimization reduced screening time.

[0027] Example 3: Heatmap Generation and Community Application The feature tensors are weighted and fused according to the weighting scheme: eye movement features focus on the clustering entropy of the gaze point, speech features strengthen the fundamental frequency perturbation coefficient, and gait features amplify the pressure center offset. The fusion vector outputs the risk index through the radial basis network, and a three-dimensional behavioral degradation heat map is generated at the same time: red marks abnormal areas of the eye movement scanning path, blue highlights speech rhythm breakpoints, and yellow marks areas of gait symmetry defects. This system has been deployed in 8 community centers, and a total of 12,000 people have been screened. The early MCI detection rate is higher than that of traditional scales, and the misdiagnosis rate is reduced.

Claims

1. A non-invasive screening system for cognitive impairment based on multimodal features, characterized by: The non-invasive screening system for cognitive impairment includes a physiological sensing terminal, a feature collaboration terminal, an intelligent decision-making terminal, a visual reporting terminal and a reinforcement learning optimization module; The physiological sensing end is used to collect pupil movement trajectories in visual tasks through the embedded eye tracker array, and the intelligent trail sensor grid to obtain the subject's walking gait dynamics in real time, and simultaneously collect eye movement trajectory data, voice interaction data and gait dynamics data; The feature coordination terminal is used to extract spatiotemporal features of the collected eye movement trajectory data, voice interaction data and gait dynamic data through the equipped spatiotemporal alignment processor and construct a feature matrix; The intelligent decision-making end is used to construct a knowledge graph through the stored clinical diagnosis rule tree, neuroimaging feature mapping table and behavior pattern association matrix. Based on this knowledge graph, a reinforcement learning decision tree generator is applied to match the optimal screening strategy and generate a screening decision path including a feature weight allocation scheme; The visualization reporting terminal is used to display the cognitive impairment risk index calculated based on the weighted fusion feature data; The reinforcement learning optimization module is used to execute the feature weight allocation scheme parsed from the decision path through a dynamic weight allocator, and iteratively improve the screening accuracy and efficiency of the decision path in the knowledge graph.

2. A non-invasive screening method for cognitive impairment based on multimodal features, characterized in that: The method comprises the following steps: S1. Use non-contact sensing equipment to synchronously collect multimodal physiological behavior data of the subject, including eye movement trajectory data, voice interaction data, and gait dynamic data; S2. Extracting spatiotemporal features from the eye movement trajectory data, voice interaction data, and gait dynamic data to generate an eye movement feature matrix, a voice feature matrix, and a gait feature matrix; S3. Aligning the temporal dimensions of the eye movement feature matrix, the speech feature matrix, and the gait feature matrix based on a dynamic time warping algorithm to construct a multimodal spatiotemporal fusion feature tensor; S4, calculating the physiological rhythm consistency index of the eye movement trajectory data and the gait dynamic data, and executing S6 when the rhythm consistency threshold meets the preset condition, otherwise executing S5; S5. Using a phase synchronization algorithm to perform rhythm resampling on the gait dynamic data, generating rhythm-calibrated gait data and updating the multimodal spatiotemporal fusion feature tensor; S6. Inputting the multimodal spatiotemporal fusion feature tensor into a pre-constructed cognitive impairment screening knowledge graph, wherein the knowledge graph comprises clinical diagnosis rule nodes, neuroimaging feature nodes, and behavioral pattern association edges; S7. Traverse the knowledge graph based on a reinforcement learning decision tree, match the optimal screening strategy and generate a screening decision path, wherein the decision path includes a feature weight allocation scheme; S8. Perform weighted fusion on the multimodal spatiotemporal fusion feature tensor according to the feature weight distribution scheme, and output a cognitive impairment risk index and a multidimensional behavior degradation heat map.

3. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: Said S1 specifically includes: S11. Use an infrared eye tracker to collect eye movement data during the visual exploration task at a sampling rate greater than 120 Hz, and record pupil position change vectors. S12. Collect voice command response data through a directional microphone array and extract fundamental frequency disturbance parameters and semantic coherence scores; S13. Gait sequences were collected using a pressure-sensing walkway at a sampling rate of 100 Hz to generate a spatiotemporal matrix of plantar pressure distribution.

4. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S2 includes: S21. Use a convolutional recurrent neural network to extract the saccade acceleration features and gaze point clustering features from the eye movement trajectory data to generate an 8-dimensional eye movement feature vector; S22. Extract acoustic fingerprint features from voice interaction data through Mel-frequency cepstral coefficient decomposition, and use a bidirectional LSTM network to generate a 16-dimensional voice time series feature matrix; S23. Wavelet packet decomposition is used to extract the trajectory characteristics of the pressure center of the support phase from the gait dynamic data, and a 12-dimensional gait dynamic parameter matrix is ​​constructed.

5. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S3 includes: S31. Construct a three-dimensional feature tensor space, whose dimensions are defined as: time axis T, feature axis F, and modal axis M; S32, using a dynamic time warping algorithm to calculate the time offset δt of the eye movement-speech feature sequence, and unifying the three modal data to the same time baseline through an interpolation algorithm; S33. Stack the aligned feature matrices according to the modal dimension to generate a multimodal spatiotemporal fusion feature tensor with a dimension of [T×36×3].

6. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S4 includes: S41. Calculate the power spectrum density of the saccade rhythm of the eye movement trajectory and the cadence power spectral density of gait data ; S42. Define rhythm consistency indicators: ； in, is an index of rhythm consistency. is the power spectral density function, is the rhythmic characteristic of the eye movement trajectory, is the rhythmic characteristic of gait dynamics, is the frequency differential element, when When ≤0.15, it was judged as consistent rhythm; S43, when When the value is >0.15, the rhythm resampling process is triggered; otherwise, feature tensor fusion is performed directly.

7. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S5 includes: S51. Constructing a gait rhythm phase space model: ； in, is the instantaneous phase of the gait rhythm, for It's time function, is a mathematical constant, is the instantaneous frequency function of gait, is the time variable, is the time element; S52, use phase-locked loop technology to reduce the eye movement scanning frequency As a reference signal, adjust the gait phase to the synchronous state; S53. Resample the gait data at the synchronous phase points using cubic spline interpolation.

8. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S6 includes: S61. The knowledge graph comprises a three-layer structure: The top layer is the clinical diagnosis rule node, which includes the MMSE score threshold and CDR grading standard; The middle layer is the neuroimaging feature node, which includes hippocampal atrophy rate and default network connection strength; The bottom layer is a multimodal behavior feature node, which is directly associated with the multimodal spatiotemporal fusion feature tensor; S62. Each node is connected by Bayesian probability edges, and the edge weights are trained by historical diagnostic data.

9. The non-invasive screening method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S7 includes: S71. Construct the state space S = {knowledge graph node, current feature weight} of the reinforcement learning decision tree; S72. Define action space A = {node jump direction, weight adjustment range}; S73. Set reward function: ； in, is the reinforcement learning reward value, is the diagnostic accuracy weight factor, To verify the accuracy of clinical is the feature coverage weight factor, is the effective feature utilization, is the path length penalty factor, is the decision path complexity; S74. Iteratively optimize the decision path through the Q-learning algorithm until convergence.

10. The non-invasive screening system and method for cognitive impairment based on multimodal features according to claim 2, characterized in that: The S8 includes: S81. Calculate the weighted feature vector according to the weight distribution scheme output by the decision path: ； in, is the weighted eigenvector, is the modal weight coefficient, is the normalization operator, is the normalized modal feature; S82, will Input the radial basis function network and output the cognitive impairment risk index in the range of 0-1; S83. Generate a three-dimensional heat map based on feature contribution, marking abnormal areas of eye movement scanning paths, speech rhythm breakpoints, and gait symmetry defect areas.

Images