Non-invasive visual rehabilitation evaluation system and method

Abstract

The invention discloses a non-intrusive visual rehabilitation evaluation system and method, which can objectively and accurately reflect the activity and response of the brain under visual stimulation by adopting a non-intrusive bimodal neural signal acquisition module and synchronously acquiring electroencephalogram and functional near infrared spectrum signals, thereby avoiding evaluation deviation caused by subjective factors of a patient. Secondly, a bimodal signal fusion processing module is used for signal enhancement preprocessing and fusion decoding, visual cortex feature rules are deeply mined, the visual function of a patient is comprehensively known, meanwhile, real-time signal collection and rapid processing are achieved through the technology, and a visual perception state evaluation result is obtained in time; besides, visual stimulation parameters are dynamically adjusted through a closed-loop parameter optimization module on the basis of comparison of an evaluation result and a preset threshold value, a closed-loop feedback mechanism is formed, training meets user requirements, training pertinence and effectiveness are enhanced, and then the visual rehabilitation training effect is improved.

Description

Technical Field

[0001] This invention relates to the field of visual rehabilitation assessment technology, and in particular to a non-invasive visual rehabilitation assessment system and method. Background Technology

[0002] Visual rehabilitation assessment is crucial for the treatment and rehabilitation of patients with visual impairments. It helps medical staff accurately understand the patient's visual function status, thereby developing personalized rehabilitation treatment plans.

[0003] Traditional visual rehabilitation assessment methods are typically represented by fixed pattern stimulus training, visual tracking training, and contrast sensitivity training devices. Their core implementation logic is: to repeatedly present a preset specific visual stimulus (such as a black and white checkerboard, moving light spots, etc.) through training equipment to guide users to perform visual focus, tracking, or discrimination exercises. During the training process, the training effect is evaluated solely based on behavioral feedback information (such as user fixation stability, subjective feeling reports, post-training visual acuity, reading speed, contrast sensitivity test data, etc.), and the parameters of subsequent training programs (such as stimulus pattern complexity, training duration, training frequency, etc.) are manually adjusted based on the evaluation results. Although traditional visual rehabilitation assessment methods can achieve certain visual rehabilitation assessment effects, they still have the following defects: (1) lack of objective evaluation basis at the neurological level: traditional visual training relies entirely on The behavioral feedback (such as vision and user subjective feelings) is used to evaluate the training effect, but it does not involve the collection and analysis of the user's visual cortex neural response signal. Since the visual function state is directly related to the brain's neural activity, the lack of objective data at the neural level leads to the evaluation results being one-sided and subjective, and easily affected by factors such as the user's cognitive level, cooperation level, and fatigue state. It cannot truly reflect the activation effect of training on the visual center, and it is also difficult to accurately judge the effectiveness of the training plan. (2) There is no dynamic feedback and real-time control mechanism: During the training process, it is impossible to obtain the user's visual perception state and physiological response data in real time, and it is impossible to achieve dynamic optimization during the training process. When the user experiences decreased attention, visual fatigue, or insensitivity to the current stimulus, it is impossible to adjust the stimulus parameters in time, which not only affects the training efficiency, but may also cause user resistance due to continuous ineffective stimulation. Summary of the Invention

[0004] In view of this, the present invention proposes a non-invasive visual rehabilitation assessment system and method, which can effectively solve the defects of existing technologies that rely on behavioral feedback to assess training effects, do not involve the collection and analysis of the user's visual cortex neural response signals, cannot obtain the user's visual perception state and physiological response data in real time, and cannot achieve dynamic optimization during the training process.

[0005] The technical solution of this invention is implemented as follows:

[0006] A non-invasive visual rehabilitation assessment system includes:

[0007] A non-invasive dual-modal neural signal acquisition module is used to simultaneously acquire electroencephalogram (EEG) signals and functional near-infrared spectral signals based on visual stimulation signals.

[0008] The dual-modal signal fusion processing module is used to perform enhanced preprocessing and fusion decoding on the acquired EEG signals and functional near-infrared spectral signals to obtain visual perception state assessment results.

[0009] The closed-loop parameter optimization module is used to dynamically calculate the parameter adjustment amount based on the visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amount.

[0010] As a further optional embodiment of the non-invasive visual rehabilitation assessment system, the non-invasive bimodal neural signal acquisition module includes:

[0011] The EEG acquisition unit is used to acquire EEG signals and perform bandpass filtering and notch filtering on the EEG signals;

[0012] A functional near-infrared spectral signal acquisition unit is used to acquire functional near-infrared spectral signals;

[0013] The triaxial accelerometer unit is used to combine a triaxial accelerometer, independent component analysis algorithm, and motion compensation model to remove motion artifacts from EEG signals and functional near-infrared spectroscopy signals.

[0014] As a further optional embodiment of the non-invasive visual rehabilitation assessment system, the triaxial acceleration sensing unit includes:

[0015] A triaxial accelerometer is used to monitor the user's head movement in real time and acquire acceleration data in three-dimensional space.

[0016] The time alignment subunit is used to time-align acceleration data with synchronously acquired EEG signals and functional near-infrared spectral signals.

[0017] The independent component identification subunit is used to perform blind source separation on EEG signals and functional near-infrared spectral signals using independent component analysis algorithms, and to identify and separate motion-related independent components.

[0018] The motion component separation subunit is used to combine with the motion compensation model to correct or remove the separated motion-related components, thereby obtaining EEG signals and functional near-infrared spectral signals with motion artifacts removed.

[0019] As a further optional solution to the aforementioned non-invasive visual rehabilitation assessment system, the dual-modal signal fusion processing module includes:

[0020] The enhanced preprocessing unit is used to enhance and preprocess the acquired EEG signals and functional near-infrared spectral signals to obtain enhanced preprocessed EEG signals and functional near-infrared spectral signals.

[0021] The feature extraction unit is used to extract steady-state visual evoked potential features from the enhanced preprocessed EEG signals, and to extract brain blood oxygen saturation features from the first visual cortex of the enhanced preprocessed functional near-infrared spectral signals.

[0022] The fusion decoding processing unit is used for the attention-based CNN-LSTM hybrid model. The inputs are steady-state visual evoked potential features and brain blood oxygen saturation features of the first visual cortex, and the output is the visual perception state assessment result.

[0023] As a further optional solution to the aforementioned non-invasive visual rehabilitation assessment system, the fusion decoding processing unit includes:

[0024] The input layer is used to receive steady-state visual evoked potential features and brain blood oxygen saturation features of the first visual cortex.

[0025] The weight allocation subunit is used to dynamically allocate the weights of steady-state visual evoked potential features and brain blood oxygen saturation features in the first visual area through an attention mechanism, and to perform feature fusion based on the weights to obtain a fusion vector sequence.

[0026] The CNN-LSTM hybrid model is used to extract spatial hierarchical features from the fused vector sequence using the CNN module, and to capture temporal hierarchical features from the fused vector sequence using the LSTM module.

[0027] The output layer is used to generate multidimensional visual perception state assessment results based on the CNN-LSTM hybrid model, including attention concentration level, stimulus recognition accuracy, steady-state visual evoked potential signal-to-noise ratio, and brain blood oxygenation activation intensity in the first visual zone.

[0028] As a further optional solution to the aforementioned non-invasive visual rehabilitation assessment system, the closed-loop parameter optimization module includes:

[0029] The parameter adjustment calculation unit is used to compare and calculate the parameter adjustment amount based on the visual perception state assessment result and the preset threshold.

[0030] The visual stimulus signal generation unit is used to generate new visual stimulus signals based on parameter adjustment using a fuzzy PID closed-loop control algorithm.

[0031] As a further optional solution to the aforementioned non-invasive visual rehabilitation assessment system, the system also includes:

[0032] The training management module is used to record training data and generate training reports;

[0033] The feedback and interaction module is used to generate visual rehabilitation assessment results based on training reports, and to present the training status and assessment results in real time.

[0034] A non-invasive visual rehabilitation assessment method, specifically including:

[0035] Using a non-invasive dual-modal neural signal acquisition module, EEG signals and functional near-infrared spectral signals are acquired simultaneously based on visual stimulation signals;

[0036] The dual-modal signal fusion processing module enhances the preprocessing and fusion decoding of the acquired EEG signals and functional near-infrared spectral signals to obtain visual perception state assessment results.

[0037] A closed-loop parameter optimization module is used to dynamically calculate the parameter adjustment amount based on the visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amount.

[0038] A computing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the non-invasive visual rehabilitation assessment method.

[0039] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the non-invasive visual rehabilitation assessment method.

[0040] The beneficial effects of this invention are as follows: By employing a non-invasive dual-modal neural signal acquisition module, both electroencephalogram (EEG) signals and functional near-infrared spectral signals are simultaneously acquired. These two signals directly reflect the brain's neural activity and physiological response to visual stimulation, and are more objective and accurate than behavioral feedback. Analyzing these neural signals to assess the effectiveness of visual rehabilitation training avoids assessment bias caused by the patient's subjective factors. Secondly, the dual-modal signal fusion processing module further enhances the preprocessing and decoding of these signals, deeply exploring the characteristics and patterns of visual cortical neural responses, which helps to more comprehensively and deeply understand the patient's visual functional state and visual information processing mechanism. Furthermore, the non-invasive dual-modal neural signal acquisition module can acquire signals in real time and synchronously. This system integrates electroencephalogram (EEG) signals and functional near-infrared spectroscopy (FIR) signals, which reflect the user's visual perception state and physiological response in real time. The dual-modal signal fusion processing module rapidly processes the acquired signals to obtain timely visual perception state assessment results. The closed-loop parameter optimization module compares the real-time visual perception state assessment results with preset thresholds, dynamically calculates parameter adjustment amounts, and generates new visual stimulus signals based on these adjustments. This closed-loop feedback mechanism can adjust the parameters of visual stimuli, such as stimulation frequency, contrast, and presentation duration, in a timely manner according to the user's current visual perception state and physiological response. This makes the training process more aligned with the user's actual needs, improves the targeting and effectiveness of training, and thus enhances the effect of visual rehabilitation training. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the composition of a non-invasive visual rehabilitation assessment system according to the present invention;

[0043] Figure 2 This is a flowchart illustrating a non-invasive visual rehabilitation assessment method according to the present invention.

[0044] Figure 3 This is a schematic diagram of the composition of a computing device according to the present invention. Detailed Implementation

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] refer to Figures 1 to 3 A non-invasive visual rehabilitation assessment system includes a non-invasive bimodal neural signal acquisition module, a bimodal signal fusion processing module, a closed-loop parameter optimization module, a training management module, and a feedback interaction module, wherein:

[0047] A non-invasive bimodal neural signal acquisition module is used to simultaneously acquire electroencephalogram (EEG) signals and functional near-infrared spectral signals based on visual stimulus signals; in some embodiments, the non-invasive bimodal neural signal acquisition module includes:

[0048] The EEG acquisition unit is used to acquire EEG signals. The electrodes precisely cover the scalp area corresponding to the occipital visual cortex (including the area above the first visual field). It has a built-in 32-channel flexible dry electrode with a sampling frequency of 500Hz and a built-in 24-bit AD converter. It has 0.1-100Hz bandpass filtering and 50Hz notch filtering functions, which can effectively remove some noise interference.

[0049] The functional near-infrared spectral signal acquisition unit is used to acquire functional near-infrared spectral signals. It uses near-infrared light (dual wavelengths of 730nm and 850nm). The detection points and light source points are distributed in pairs in the corresponding area of ​​the scalp in the first visual zone. The sampling frequency is 10Hz, and it is used to acquire the cerebral blood oxygen saturation in the first visual zone. Signals (such as HbR concentration changes) are used to reflect the metabolic state of the visual cortex;

[0050] The triaxial accelerometer unit combines a triaxial accelerometer, independent component analysis algorithm, and motion compensation model to remove motion artifacts from EEG and functional near-infrared spectroscopy signals, ensuring the stability of signal acquisition. It eliminates the need for conductive gel, thus improving user comfort.

[0051] Specifically, the patient is fitted with a data acquisition device equipped with an EEG acquisition unit, a functional near-infrared spectroscopy signal acquisition unit, and a triaxial accelerometer. The 32-channel flexible dry electrodes of the EEG acquisition unit are precisely positioned over the scalp area corresponding to the occipital visual cortex (including the area above the first visual cortex), ensuring good contact between the electrodes and the scalp. The detection points and light source points of the functional near-infrared spectroscopy signal acquisition unit are paired and distributed in the scalp area corresponding to the first visual cortex. The triaxial accelerometer is fixed at a suitable position on the patient's head. Based on the assessment requirements, the sampling frequency of the EEG acquisition unit is set to 500Hz, and its built-in 0.1-100Hz bandpass filter and 50Hz notch filter are enabled to remove environmental noise and power frequency interference. The functional near-infrared spectroscopy signal acquisition unit uses dual-wavelength near-infrared light of 730nm and 850nm, with a sampling frequency set to 10Hz, to acquire cerebral blood oxygen saturation in the first visual cortex. The system acquires signals from various sources, including HbR concentration changes. Specific visual stimuli are presented to the patient, such as flashes or patterns of varying frequencies, contrasts, and patterns. Simultaneously, the EEG acquisition unit, functional near-infrared spectroscopy (FIR) signal acquisition unit, and triaxial accelerometer unit begin synchronous signal acquisition. The EEG acquisition unit acquires the patient's EEG signals in real time, the FIR signal acquisition unit continuously acquires cerebral blood oxygen saturation signals in the first visual cortex, and the triaxial accelerometer unit monitors the patient's head movements. Using independent component analysis (ICA) algorithms and motion compensation models, combined with data acquired by the triaxial accelerometer unit, the EEG and FIR signals are analyzed to identify and remove motion artifacts, ensuring the stability of signal acquisition.

[0052] Thus, the 32-channel flexible dry electrode precisely covers the relevant areas of the occipital visual cortex, enabling comprehensive and detailed acquisition of EEG activity information in this region. The high sampling frequency of 500Hz ensures the temporal resolution of the signal, capturing details of rapidly changing EEG signals. Built-in bandpass and notch filtering effectively removes noise interference, improving the quality and signal-to-noise ratio of the EEG signal. The use of dual-wavelength near-infrared light allows for accurate measurement of cerebral oxygen saturation in the first visual cortex. The device acquires signals (including HbR concentration changes) reflecting the metabolic state of the visual cortex. A 10Hz sampling frequency meets signal acquisition requirements while ensuring real-time data transmission. Secondly, a triaxial accelerometer unit, combined with independent component analysis algorithms and a motion compensation model, can monitor patient head movements in real time and accurately identify and remove motion artifacts from EEG and functional near-infrared spectroscopy signals. This significantly improves signal stability and reliability, avoiding signal distortion caused by patient head movements and providing a high-quality data foundation for subsequent signal analysis and processing. Furthermore, the use of flexible dry electrodes eliminates the need for conductive gel, reducing discomfort and improving patient acceptance and comfort. The integrated design and convenient operation of the device make signal acquisition simpler and more efficient. The simultaneous operation of the three acquisition units enables the synchronous acquisition of EEG signals, functional near-infrared spectroscopy signals, and head movement signals.

[0053] In some embodiments, the triaxial acceleration sensing unit includes:

[0054] A triaxial accelerometer is used to monitor the user's head movement in real time and acquire acceleration data in three-dimensional space.

[0055] The time alignment subunit is used to time-align acceleration data with synchronously acquired EEG signals and functional near-infrared spectral signals.

[0056] The independent component identification subunit is used to perform blind source separation on EEG signals and functional near-infrared spectral signals using independent component analysis algorithms, and to identify and separate motion-related independent components.

[0057] The motion component separation subunit is used to combine with the motion compensation model to correct or remove the separated motion-related components, thereby obtaining EEG signals and functional near-infrared spectral signals with motion artifacts removed.

[0058] Specifically, a triaxial accelerometer is securely and precisely mounted on the subject's head to ensure accurate perception of head movement. Simultaneously, an EEG acquisition device and a functional near-infrared spectroscopy (FIR) signal acquisition device are installed at corresponding positions on the subject's head to synchronously acquire EEG and FIR signals. Parameter settings are configured for the time alignment subunit, independent component recognition subunit, and motion component separation subunit. For example, appropriate time alignment parameters are set based on the sampling frequencies of the EEG and FIR signals to ensure accurate time-axis correspondence between acceleration data and these two signals. During the subject's reception of specific visual stimuli, the triaxial accelerometer monitors the subject's head acceleration changes in three-dimensional space in real time and transmits the data to the time alignment subunit. Simultaneously, the EEG acquisition device and the FIR signal acquisition device... The spectral signal acquisition device acquires the corresponding signals and transmits them to the subsequent processing unit. The time alignment subunit, based on preset parameters, precisely aligns the received acceleration data with the synchronously acquired EEG and functional near-infrared spectral signals to ensure consistency of the three signals in the time dimension. The independent component identification subunit uses an independent component analysis algorithm to perform blind source separation on the aligned EEG and functional near-infrared spectral signals. By analyzing the statistical characteristics of the signals, it identifies and separates motion-related independent components. The motion component separation subunit, combined with a motion compensation model, further analyzes and processes the motion-related components separated by the independent component identification subunit. Based on model calculations, it corrects or directly removes the motion-related components in the EEG and functional near-infrared spectral signals, ultimately obtaining a clean signal free of motion artifacts.

[0059] Thus, the triaxial accelerometer can acquire real-time and accurate acceleration data of the head in three-dimensional space, providing a precise motion information foundation for subsequent signal processing. This helps to comprehensively understand the impact of head movement on neural signal acquisition and improve the targeting of motion artifact removal. The time alignment subunit ensures strict synchronization of acceleration data with EEG signals and functional near-infrared spectroscopy signals in time. This is crucial for accurately analyzing the temporal relationship between head movement and changes in neural signals, avoiding erroneous analysis and incomplete artifact removal caused by time deviations. The independent component identification subunit uses an independent component analysis algorithm, which can effectively identify and separate motion-related independent components from complex EEG and functional near-infrared spectroscopy signals. This method does not require prior knowledge of the specific source and mixing mode of the signal, has strong adaptability and versatility, and improves the efficiency and accuracy of motion artifact separation. The motion component separation subunit, combined with a motion compensation model, performs fine processing on the separated motion-related components, which can correct or remove motion artifacts in EEG and functional near-infrared spectroscopy signals to the greatest extent. The processed signal is purer and more realistically reflects the neural activity and physiological response of the brain under visual stimulation.

[0060] A dual-modal signal fusion processing module is used to perform enhanced preprocessing and fusion decoding on the acquired electroencephalogram (EEG) signals and functional near-infrared spectral signals to obtain visual perception state assessment results; in some embodiments, the dual-modal signal fusion processing module includes:

[0061] The enhancement preprocessing unit is used to enhance and preprocess the acquired EEG signals and functional near-infrared spectral signals to obtain enhanced preprocessed EEG signals and functional near-infrared spectral signals. The enhancement preprocessing includes bandpass filtering, detrending, and lowpass filtering.

[0062] The feature extraction unit is used to extract steady-state visual evoked potential features from the enhanced preprocessed EEG signals, and to extract brain blood oxygen saturation features from the first visual cortex of the enhanced preprocessed functional near-infrared spectral signals.

[0063] The fusion decoding processing unit is used for the attention-based CNN-LSTM hybrid model. The inputs are steady-state visual evoked potential features and brain blood oxygen saturation features of the first visual cortex, and the output is the visual perception state assessment result.

[0064] Specifically, the EEG signals and functional near-infrared spectral signals acquired by the non-invasive bimodal neural signal acquisition module are input into the enhancement preprocessing unit. For the EEG signals, the enhancement preprocessing unit first uses a bandpass filter (such as a 0.1-100Hz bandpass filter) to filter the EEG signals, removing high-frequency noise and low-frequency drift, and retaining the effective frequency components related to visual perception. At the same time, a detrending algorithm is used to eliminate the linear trend in the signal, further improving the signal quality. For the functional near-infrared spectral signals, a low-pass filter (such as a low-pass filter with a cutoff frequency of 2Hz) is used to remove high-frequency noise, resulting in a smoother signal for subsequent processing.

[0065] The enhanced preprocessed EEG signal is input into the feature extraction unit, which focuses on extracting steady-state visual evoked potentials (SSVEP) features. Using methods such as Fourier transform, it analyzes the energy distribution of the EEG signal within a specific frequency range, extracting SSVEP feature parameters related to the visual stimulus frequency, such as frequency, amplitude, and signal-to-noise ratio. Simultaneously, the enhanced preprocessed functional near-infrared spectral signal also enters the feature extraction unit to extract brain oxygen saturation features in the first visual cortex (V1 area). Through specific algorithms, it is calculated to obtain... The characteristics of changes in the concentrations of oxyhemoglobin (HbR) and deoxyhemoglobin (HbR) can reflect the metabolic activity of the V1 region under visual stimulation.

[0066] The extracted steady-state visual evoked potential features and brain blood oxygen saturation features from the first visual cortex are input into the fusion decoding processing unit. The fusion decoding processing unit processes the data using a CNN-LSTM hybrid model based on an attention mechanism. The CNN part first performs convolution operations on the input features to extract the spatial hierarchical relationships between the features. Then, it dynamically assigns weights to different features (SSVEP features and brain blood oxygen saturation features) through an attention mechanism to highlight the feature information that is more critical to the assessment of visual perception state. Next, the LSTM part performs time series analysis on the feature sequence with attention weights to capture the change pattern of features over time. Finally, the model outputs the visual perception state assessment results, such as the degree of attention concentration and the accuracy of stimulus recognition.

[0067] In some embodiments, the fusion decoding processing unit includes:

[0068] The input layer is used to receive steady-state visual evoked potential features (including key parameters such as frequency, amplitude, and signal-to-noise ratio) and brain blood oxygen saturation features of the first visual cortex (covering...). HbR concentration variation characteristics).

[0069] The weight allocation subunit is used to employ an attention mechanism to dynamically allocate the weights of steady-state visual evoked potential features and brain blood oxygen saturation features in the first visual area to assess the visual perception state. The two features are then fused by weighted summation to obtain a fused vector sequence, thereby highlighting the contribution of key features to the decoding results.

[0070] The CNN-LSTM hybrid model consists of a CNN module and an LSTM module. The CNN module uses convolutional kernels to perform sliding operations on the fused vector sequence to extract spatial hierarchical features and capture the spatial relationships between different features. The LSTM module, through its unique gating mechanism, effectively handles the temporal dependencies in the fused vector sequence, captures the dynamic changes of the time series, and thus more accurately understands the evolution of visual perception states.

[0071] The output layer is used to generate visual perception state assessment results based on the CNN-LSTM hybrid model, which includes multi-dimensional indicators such as attention concentration level (0-100 points), stimulus recognition accuracy (%), steady-state visual evoked potential signal-to-noise ratio, and brain blood oxygenation activation intensity in the first visual zone.

[0072] Specifically, the steady-state visual evoked potential characteristics (including key parameters such as frequency of 10Hz, amplitude of 5μV, and signal-to-noise ratio of 15dB) and brain blood oxygen saturation characteristics of the first visual cortex (covering...) were collected using specialized equipment in the early stage. The changes (such as a 2 μmol / L increase in HbR concentration and a 1 μmol / L decrease in HbR concentration) are input to the input layer of the fusion decoding processing unit.

[0073] The weight allocation subunit initiates an attention mechanism, dynamically analyzing the importance of steady-state visual evoked potential features and brain oxygen saturation features in the first visual cortex for assessing visual perception status. For example, when assessing immediate responses to visual stimuli, steady-state visual evoked potential features may be more critical, thus receiving a higher weight. Conversely, when assessing the ongoing metabolic effects of visual stimuli, the weight of brain oxygen saturation features in the first visual cortex increases accordingly. By weighted summation, the two features are fused into a fusion vector sequence, highlighting the contribution of key features to subsequent decoding. This makes the fused vector sequence more accurately reflect the core information of visual perception status, improving the rationality and effectiveness of feature fusion.

[0074] The fused vector sequence is input into the CNN-LSTM hybrid model. The CNN module uses convolutional kernels to slide across the fused vector sequence, such as using 3×3 convolutional kernels, to extract spatial hierarchical features and capture the spatial correlation between steady-state visual evoked potential features and cerebral blood oxygen saturation features, such as the relationship between the two when reflecting the activity of different areas of the visual cortex. The LSTM module, through its unique gating mechanism (input gate, forget gate, output gate), effectively handles the temporal dependencies in the fused vector sequence. For example, it can remember the visual perception state information of previous moments and combine it with the fused vector at the current moment to capture the dynamic changes of the visual perception state over time, such as the fluctuation of attention concentration at different time points.

[0075] The output layer, based on the processing results of the CNN-LSTM hybrid model, generates data including attention level (represented by a score of 0-100; in this experiment, the subject's attention level was 75), stimulus recognition accuracy (85%), steady-state visual evoked potential signal-to-noise ratio (maintained at approximately 15 dB), and brain oxygenation activation intensity in the first visual cortex according to preset rules. The visual perception state assessment results include multiple indicators such as concentration changes reflecting a moderate level of activation intensity.

[0076] A closed-loop parameter optimization module is used to dynamically calculate parameter adjustment amounts based on visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amounts; in some embodiments, the closed-loop parameter optimization module includes:

[0077] The parameter adjustment calculation unit is used to acquire the visual perception state assessment results in real time and compare them in detail with the preset threshold range. By analyzing the degree of deviation and trend of the assessment results from the threshold, the parameter adjustment amount is accurately calculated using calculation rules (such as difference calculation, ratio calculation, etc.). The parameter adjustment amount covers multiple key parameters of the visual stimulus signal, such as stimulus frequency, contrast, presentation duration, difficulty level, etc., to ensure that the visual stimulus can be adjusted comprehensively and effectively to meet the rehabilitation needs of different users.

[0078] The visual stimulus signal generation unit employs a fuzzy PID closed-loop control algorithm. This algorithm combines the fuzzy inference capability of fuzzy control with the precise adjustment characteristics of PID control. Taking parameter adjustment as input, it calculates and generates new visual stimulus signals in real time according to preset fuzzy rules and PID parameters (proportional coefficient, integral coefficient, and derivative coefficient). The new visual stimulus signals can be dynamically adjusted according to changes in the user's visual perception state, thereby achieving personalized visual rehabilitation training and improving training effectiveness and rehabilitation efficiency.

[0079] Specifically, during training, the parameter adjustment calculation unit acquires real-time visual perception state assessment results. For example, through the previous dual-modal signal fusion processing module, assessment indicators such as the patient's attention concentration level to the current visual stimulus signal being 60 points (out of 100) and the stimulus recognition accuracy rate being 70% are obtained. These assessment results are then compared in detail with preset threshold ranges. Assuming the preset attention concentration threshold range is 70-100 points and the stimulus recognition accuracy threshold range is 80-100%, analysis reveals that the attention concentration level is 10 points below the lower threshold and the stimulus recognition accuracy rate is 10% below the lower threshold. Using rules such as difference calculation, the parameter adjustment amount is precisely calculated. For the stimulus frequency, if the current frequency is 10Hz, based on the empirical model and calculation rules, to improve the patient's attention and recognition accuracy, the adjustment amount is determined to be an increase of 2Hz; the contrast adjustment amount is set to increase by 15%; the presentation duration adjustment amount is set to extend by 0.5s; and the difficulty level adjustment amount is set to decrease by 1 level, etc., to ensure that the visual stimulus can be adjusted comprehensively and effectively.

[0080] The visual stimulus signal generation unit employs a fuzzy PID closed-loop control algorithm. Using the calculated parameter adjustment amount as input, it calculates and generates new visual stimulus signals in real time based on preset fuzzy rules and PID parameters (proportional coefficient, integral coefficient, and derivative coefficient). For example, when adjusting the stimulus frequency, the fuzzy rules use the difference between the current frequency and the target frequency, along with the rate of change of that difference, combined with the PID parameters to perform fuzzy inference and precise calculation to determine the final frequency adjustment value. The new visual stimulus signal can be dynamically adjusted according to changes in the user's visual perception state. For instance, when the patient's attention to the current stimulus gradually increases, the system automatically increases the difficulty or frequency of the stimulus to further stimulate the development and improvement of the visual system. Conversely, if the patient experiences fatigue or decreased attention, the system appropriately reduces the intensity or difficulty of the stimulus to achieve personalized visual rehabilitation training.

[0081] In this way, multiple key parameters of the visual stimulus signal, such as stimulation frequency and contrast, can be precisely adjusted according to the patient's actual performance, ensuring that the visual stimulus is always in the most suitable state for the patient's rehabilitation, thereby improving the pertinence and effectiveness of rehabilitation training. At the same time, the stimulus signal can be quickly and accurately adjusted according to the dynamic changes in the patient's visual perception state, realizing personalized visual rehabilitation training and avoiding the limitation of a static stimulus pattern on the patient's rehabilitation effect.

[0082] In some embodiments, the system further includes:

[0083] The training management module is used to record training data and generate training reports;

[0084] The feedback and interaction module is used to generate visual rehabilitation assessment results based on training reports, and to present the training status and assessment results in real time.

[0085] Specifically, before training begins, staff enter the patient's basic information, such as name, age, and type of amblyopia, into the training management module, and develop a personalized training plan for the patient, including training items, duration of each training session, and training frequency. During training, the training management module records the patient's training data in real time. For example, when the patient is undergoing visual stimulus training, it records detailed data such as reaction time, number of correct responses, and number of incorrect responses for each different stimulus pattern. After training, the training management module automatically generates a training report based on the recorded training data. The report includes the patient's basic information, the implementation status of the training plan, statistics of various data during the training process (such as average reaction time, overall accuracy rate, etc.), and records of any abnormal situations during the training process.

[0086] The feedback interaction module receives training reports generated by the training management module. Based on the data in the reports, it uses a preset evaluation algorithm to generate visual rehabilitation assessment results. For example, it assesses the improvement of the patient's visual function, such as the degree of visual acuity improvement and changes in contrast sensitivity, based on changes in the patient's training data over a period of time. This module presents the assessment results in real time in the form of intuitive charts (such as bar charts showing visual acuity values ​​at different stages and line graphs showing the trend of reaction time changes) and text descriptions. Patients and medical staff can view the training status and assessment results through the system's display interface to understand the patient's rehabilitation progress. At the same time, patients can also communicate with medical staff based on the presented results, and medical staff can further adjust the training plan based on the feedback.

[0087] A non-invasive visual rehabilitation assessment method, specifically including:

[0088] Using a non-invasive dual-modal neural signal acquisition module, EEG signals and functional near-infrared spectral signals are acquired simultaneously based on visual stimulation signals;

[0089] The dual-modal signal fusion processing module enhances the preprocessing and fusion decoding of the acquired EEG signals and functional near-infrared spectral signals to obtain visual perception state assessment results.

[0090] A closed-loop parameter optimization module is used to dynamically calculate the parameter adjustment amount based on the visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amount.

[0091] A computing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the non-invasive visual rehabilitation assessment method.

[0092] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the non-invasive visual rehabilitation assessment method.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-invasive visual rehabilitation assessment system, characterized in that, include: A non-invasive dual-modal neural signal acquisition module is used to simultaneously acquire electroencephalogram (EEG) signals and functional near-infrared spectral signals based on visual stimulation signals. The dual-modal signal fusion processing module is used to perform enhanced preprocessing and fusion decoding on the acquired EEG signals and functional near-infrared spectral signals to obtain visual perception state assessment results. The closed-loop parameter optimization module is used to dynamically calculate the parameter adjustment amount based on the visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amount.

2. The non-invasive visual rehabilitation assessment system according to claim 1, characterized in that, The non-invasive dual-modal neural signal acquisition module includes: The EEG acquisition unit is used to acquire EEG signals and perform bandpass filtering and notch filtering on the EEG signals; A functional near-infrared spectral signal acquisition unit is used to acquire functional near-infrared spectral signals; The triaxial accelerometer unit is used to combine a triaxial accelerometer, independent component analysis algorithm, and motion compensation model to remove motion artifacts from EEG signals and functional near-infrared spectroscopy signals.

3. The non-invasive visual rehabilitation assessment system according to claim 2, characterized in that, The triaxial acceleration sensing unit includes: A triaxial accelerometer is used to monitor the user's head movement in real time and acquire acceleration data in three-dimensional space. The time alignment subunit is used to time-align acceleration data with synchronously acquired EEG signals and functional near-infrared spectral signals. The independent component identification subunit is used to perform blind source separation on EEG signals and functional near-infrared spectral signals using independent component analysis algorithms, and to identify and separate motion-related independent components. The motion component separation subunit is used to combine with the motion compensation model to correct or remove the separated motion-related components, thereby obtaining EEG signals and functional near-infrared spectral signals with motion artifacts removed.

4. The non-invasive visual rehabilitation assessment system according to claim 3, characterized in that, The dual-mode signal fusion processing module includes: The enhanced preprocessing unit is used to enhance and preprocess the acquired EEG signals and functional near-infrared spectral signals to obtain enhanced preprocessed EEG signals and functional near-infrared spectral signals. The feature extraction unit is used to extract steady-state visual evoked potential features from the enhanced preprocessed EEG signals, and to extract brain blood oxygen saturation features from the first visual cortex of the enhanced preprocessed functional near-infrared spectral signals. The fusion decoding processing unit is used for the attention-based CNN-LSTM hybrid model. The inputs are steady-state visual evoked potential features and brain blood oxygen saturation features of the first visual cortex, and the output is the visual perception state assessment result.

5. The non-invasive visual rehabilitation assessment system according to claim 4, characterized in that, The fusion decoding processing unit includes: The input layer is used to receive steady-state visual evoked potential features and brain blood oxygen saturation features of the first visual cortex. The weight allocation subunit is used to dynamically allocate the weights of steady-state visual evoked potential features and brain blood oxygen saturation features in the first visual area through an attention mechanism, and to perform feature fusion based on the weights to obtain a fusion vector sequence. The CNN-LSTM hybrid model is used to extract spatial hierarchical features from the fused vector sequence using the CNN module, and to capture temporal hierarchical features from the fused vector sequence using the LSTM module. The output layer is used to generate multidimensional visual perception state assessment results based on the CNN-LSTM hybrid model, including attention concentration level, stimulus recognition accuracy, steady-state visual evoked potential signal-to-noise ratio, and brain blood oxygenation activation intensity in the first visual zone.

6. The non-invasive visual rehabilitation assessment system according to claim 5, characterized in that, The closed-loop parameter optimization module includes: The parameter adjustment calculation unit is used to compare and calculate the parameter adjustment amount based on the visual perception state assessment result and the preset threshold. The visual stimulus signal generation unit is used to generate new visual stimulus signals based on parameter adjustment using a fuzzy PID closed-loop control algorithm.

7. The non-invasive visual rehabilitation assessment system according to claim 6, characterized in that, The system also includes: The training management module is used to record training data and generate training reports; The feedback and interaction module is used to generate visual rehabilitation assessment results based on training reports, and to present the training status and assessment results in real time.

8. A non-invasive visual rehabilitation assessment method, characterized in that, Specifically, it includes: Using a non-invasive dual-modal neural signal acquisition module, EEG signals and functional near-infrared spectral signals are acquired simultaneously based on visual stimulation signals; The dual-modal signal fusion processing module enhances the preprocessing and fusion decoding of the acquired EEG signals and functional near-infrared spectral signals to obtain visual perception state assessment results. A closed-loop parameter optimization module is used to dynamically calculate the parameter adjustment amount based on the visual perception state assessment results, and generate new visual stimulus signals based on the parameter adjustment amount.

9. A computing device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the non-invasive visual rehabilitation assessment method of claim 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the non-invasive visual rehabilitation assessment method of claim 8.

Images