Method and system for analyzing and evaluating language of stroke patient and electronic equipment

Abstract

The invention discloses a stroke patient language analysis evaluation method and system and electronic equipment, and relates to the technical field of data processing.The method comprises the steps that electroencephalogram signals of a stroke patient are collected through non-invasive electroencephalogram signal collecting equipment, and the electroencephalogram signals comprise steady-state visual evoked potential signals; performing preliminary analysis on the steady-state visual evoked potential signal through a filter bank canonical correlation analysis algorithm, and outputting first analysis data; acquiring facial expression sensing data and head motion sensing data of the stroke patient through a multi-modal sensing device, optimizing the first analysis data according to the facial expression sensing data and the head motion sensing data, and outputting second analysis data; and inputting the second analysis data into a voice output module for voice conversion output, thereby achieving the technical effects of improving the evaluation accuracy of the expression intention of the patient and improving the communication ability and participation intention of the patient.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a language analysis and evaluation method, system and electronic equipment for stroke patients. Background Art

[0002] Stroke is a common neurological disease that often causes language dysfunction in patients, seriously affecting their quality of life. Traditional language rehabilitation methods often rely on face-to-face communication and professional speech therapy. Patients face communication difficulties during the rehabilitation process, which limits their rehabilitation effect. Although existing brain-computer interface technology has made certain progress in certain areas, it usually lacks the ability to accurately capture and analyze patient intentions, resulting in low communication efficiency. In addition, traditional language rehabilitation systems are often unable to implement personalized rehabilitation plans and lack real-time monitoring of patients' emotions and status, making it impossible to effectively improve patients' willingness to participate and rehabilitation effects. There are technical problems such as poor language analysis accuracy, which affects communication efficiency and patient willingness to participate. Summary of the Invention

[0003] The present invention provides a method, system and electronic device for language analysis and assessment of stroke patients, so as to solve the technical problems in the prior art of poor language analysis accuracy, which affects communication efficiency and patient participation willingness, and achieve the technical effect of improving the accuracy of assessment of patients' expression intentions, improving patients' communication ability and participation willingness.

[0004] In a first aspect, the present invention provides a method for language analysis and assessment of stroke patients, wherein the method comprises: The electroencephalogram (EEG) signals of stroke patients are collected by non-invasive EEG signal collection equipment, wherein the EEG signals include steady-state visual evoked potential (VSEP) signals.

[0005] The steady-state visual evoked potential signal is preliminarily analyzed using a filter bank canonical correlation analysis algorithm to output first analysis data.

[0006] The facial expression sensing data and the head movement sensing data of the stroke patient are collected by a multimodal sensing device, the first parsed data are optimized according to the facial expression sensing data and the head movement sensing data, and the second parsed data are output.

[0007] The second parsed data is input into a voice output module for voice conversion and output.

[0008] In a second aspect, the present invention further provides a language analysis and assessment system for stroke patients, wherein the system comprises: The EEG signal acquisition module is used to acquire EEG signals from stroke patients using non-invasive EEG signal acquisition equipment, wherein the EEG signals include steady-state visual evoked potential signals.

[0009] The preliminary analysis module is used to perform preliminary analysis on the steady-state visual evoked potential signal by using a filter bank canonical correlation analysis algorithm and output first analysis data.

[0010] The multimodal data optimization module is used to collect facial expression sensing data and head movement sensing data of the stroke patient through a multimodal sensing device, optimize the first parsed data according to the facial expression sensing data and the head movement sensing data, and output second parsed data.

[0011] The voice output module is used to input the second parsed data into the voice output module for voice conversion and output.

[0012] In a third aspect, the present invention further provides an electronic device comprising: a memory for storing executable instructions; and a processor for implementing a language analysis and assessment method for stroke patients provided by the present invention when executing the executable instructions stored in the memory.

[0013] The present invention discloses a method, system and electronic device for language analysis and evaluation of stroke patients, comprising: using a non-invasive EEG signal acquisition device to acquire EEG signals of stroke patients, including steady-state visual evoked potential signals; performing preliminary analysis on the acquired steady-state visual evoked potential signals through a filter group canonical correlation analysis algorithm to generate first analysis data; using a multimodal sensing device to acquire facial expression sensing data and head movement sensing data of the patient, and optimizing the first analysis data based on these data to obtain second analysis data; inputting the optimized second analysis data into a speech output module, completing speech conversion and outputting the data. The method, system and electronic device for language analysis and evaluation of stroke patients disclosed by the present invention solve the technical problems of poor language analysis accuracy, affecting communication efficiency and patient participation willingness, and achieve the technical effect of improving the accuracy of assessment of patient expression intentions, improving patients' communication ability and participation willingness. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a method for language analysis and assessment of stroke patients according to the present invention; Figure 2 This is a structural diagram of a language analysis and evaluation system for stroke patients according to the present invention; Figure 3 Schematic diagram of the structure of an exemplary electronic device of the present invention.

[0015] Explanation of the accompanying symbols: EEG signal acquisition module 11, preliminary analysis module 12, multimodal data optimization module 13, speech output module 14, processor 31, memory 32, input device 33, output device 34. DETAILED DESCRIPTION

[0016] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention, rather than all, are shown in the drawings.

[0017] Example 1 Figure 1 The figure is a flow chart of a method for language analysis and assessment of stroke patients according to the present invention, wherein the method comprises: S100: Collecting EEG signals from a stroke patient using a non-invasive EEG signal acquisition device, wherein the EEG signals include steady-state visual evoked potential signals.

[0018] Specifically, first, EEG signals of target stroke patients are collected based on non-invasive EEG signal acquisition equipment, so as to safely and effectively obtain relevant information on the patient's brain activity, provide basic data for subsequent language analysis, and avoid causing additional harm to the patient's body.

[0019] Specifically, non-invasive EEG signal acquisition equipment refers to equipment used to collect EEG signals from the surface of the human body (such as a conductive electrode cap). It usually uses electrode patches or sensors placed on the scalp to obtain EEG signals by detecting the electrical activity of the cerebral cortex. Among them, the steady-state visual evoked potential signal (SSVEP) is a special EEG signal. When the subject is exposed to external visual stimulation that flickers at a constant frequency, the cerebral cortex will produce rhythmic electrical activity synchronized with the stimulation frequency. This steady-state visual evoked potential signal has a high signal-to-noise ratio and is easy to detect.

[0020] Preferably, the above-mentioned non-invasive EEG signal acquisition device uses biocompatible materials to ensure the comfort and safety of the patient, and will not cause discomfort even if worn for a long time. The electrode arrangement is set according to the 10-20 international standard.

[0021] Using non-invasive EEG signal acquisition equipment, multiple electrodes are placed on the patient's scalp in a specific layout. These electrodes can detect weak electrical signals from the cerebral cortex. When the patient is exposed to visual stimulation, such as flashing light or graphics, the brain will produce SSVEP signals. These signals are captured and recorded by the acquisition equipment to generate corresponding EEG signals, providing raw data for subsequent signal analysis and evaluation.

[0022] In some embodiments, the EEG signal further includes an Alpha wave signal; The signal strength of the Alpha wave signal is extracted based on frequency domain analysis; a signal strength threshold is set, and the Alpha wave signal is judged according to the signal strength threshold; if the signal strength of the Alpha wave signal is greater than or equal to the signal strength threshold, the steady-state visual evoked potential signal is collected; if the signal strength of the Alpha wave signal is less than the signal strength threshold, the collection of the steady-state visual evoked potential signal is stopped, and the non-invasive EEG signal collection device is placed in a standby state.

[0023] Specifically, the EEG signals measured in normal adults when awake mainly include theta waves, alpha waves and beta waves. Among them, the Alpha (α) wave signal is an EEG signal in the frequency range of 8-13 Hz, which is generally believed to be related to the brain's relaxation state and visual processing.

[0024] Specifically, through frequency domain analysis, the Alpha wave signal can be converted from the time domain to the frequency domain, thereby revealing the frequency components and spectral characteristics of the signal, wherein the signal strength threshold is a preset value, that is, the expected signal energy, which is used to determine whether the signal strength meets the acquisition standard, thereby deciding whether to perform subsequent signal processing. For example, if the signal strength of the Alpha wave signal is greater than or equal to the signal strength threshold, it can be considered that the target patient is performing corresponding EEG activity at this time, and the steady-state visual evoked potential signal is collected; correspondingly, if the signal strength of the Alpha wave signal is less than the signal strength threshold, it indicates that the patient's condition may be unsuitable, and the steady-state visual evoked potential signal collection is stopped, and the non-invasive EEG signal collection equipment enters standby mode to avoid invalid collection.

[0025] The above steps monitor the intensity of the Alpha wave signal to determine whether the patient is in a state suitable for SSVEP signal acquisition, thereby improving the accuracy and efficiency of signal acquisition, reducing unnecessary signal acquisition to save energy consumption of the equipment, and providing higher quality data for subsequent signal analysis and evaluation.

[0026] In some implementations, the method for setting the signal strength threshold includes: Acquire an Alpha wave sample signal from the stroke patient; perform a signal intensity change sensitivity analysis on the Alpha wave sample signal, output a sensitivity index, and set a signal intensity threshold for the stroke patient according to the sensitivity index.

[0027] Specifically, a signal strength threshold is set. First, the Alpha wave sample signal of the stroke patient is obtained. The Alpha wave sample signal is collected by a non-invasive EEG signal acquisition device in different states (such as relaxation, concentration, etc.); then, the Alpha wave sample signal is subjected to a signal strength change sensitivity analysis to evaluate the sensitivity of the Alpha wave signal strength to changes in different EEG activity states, thereby quantifying the signal characteristics and outputting a sensitivity index, which reflects the impact of signal strength changes on system performance.

[0028] Exemplarily, a sensitivity analysis of signal intensity changes is performed. First, the intensity distribution of the Alpha wave signal sample in the frequency domain characteristics is calculated, and statistical characteristics such as the average intensity and peak intensity of the key frequency bands are extracted; then, the coefficient of variation of the signal under different states (such as the ratio of the standard deviation to the mean) is analyzed, and the distribution range of the signal intensity is evaluated based on a signal distribution model such as a Gaussian distribution or a normal distribution, so as to determine the amplitude of the signal intensity change, stability indicators, etc. as sensitivity indicators.

[0029] Specifically, based on sensitivity indicators, personalized signal strength thresholds are set for stroke patients to ensure the accuracy and reliability of signal acquisition, while reducing unnecessary signal acquisition and improving the overall performance of the system. For example, sensitivity analysis reveals that when the alpha wave signal strength falls below a certain value, the acquisition quality of the SSVEP signal significantly decreases. This value can be set as the threshold to optimize the signal acquisition process.

[0030] By acquiring Alpha wave sample signals from stroke patients and performing a sensitivity analysis on signal intensity changes, the above steps can set a personalized signal intensity threshold, thereby more accurately capturing the patient's language intentions and providing higher-quality data for subsequent signal analysis and evaluation, thereby helping to improve the performance and reliability of language analysis.

[0031] S200: Preliminary analysis of the steady-state visual evoked potential signal is performed using a filter bank canonical correlation analysis algorithm, and first analysis data is output.

[0032] Specifically, the filter bank canonical correlation analysis algorithm is a signal processing method that combines filter banks and canonical correlation analysis (CCA). Bandpass filters allow signals within a specific frequency range to pass through while suppressing signals at other frequencies. Canonical correlation analysis (CCA) is used to study the linear correlation between two sets of variables, reflecting the linear correlation properties between the two sets of variables by extracting canonical variable pairs. Specifically, preliminary analysis of steady-state visual evoked potential signals through the filter bank canonical correlation analysis algorithm can more accurately extract language-related feature information from complex EEG signals, providing a more reliable basis for subsequent optimization and speech output, and improving the accuracy and reliability of language analysis.

[0033] In some embodiments, performing a preliminary analysis on the steady-state visual evoked potential signal using a filter bank canonical correlation analysis algorithm and outputting first analysis data, the method includes: Among them, the filter group includes multiple band-pass filters, and the steady-state visual evoked potential signal is decomposed into multiple frequency bands according to the multiple band-pass filters to extract multiple decomposition signals; multiple language template signals are obtained, and the multiple language template signals correspond to the multiple decomposition signals; the multiple decomposition signals are preliminarily analyzed based on the multiple language template signals according to the typical correlation analysis algorithm, and first analysis data is output.

[0034] Specifically, the filter group includes multiple bandpass filters for multi-band decomposition of the signal; the language template signal is a set of predefined signals used to compare and analyze the decomposed signal, so as to identify and understand specific patterns or information in the signal. Exemplarily, the language template signal includes the patient's brain wave signal in a specific language stimulation scenario (such as reading specific text or viewing specific images). The brain wave signal generation is achieved by recording the brain wave signal of stroke patients when facing specific language stimulation (such as displaying images or text such as "apple" and "house" under experimental conditions).

[0035] Specifically, first, multiple band-pass filters are designed based on the target frequency and harmonics of the steady-state visual evoked potential (SSVEP) signal, where each filter corresponds to a frequency band (such as 8-30 Hz) and is used to extract signal components within different frequency ranges; then, the collected SSVEP signal is decomposed into multiple frequency bands using the designed filter group, and multiple decomposition signals are output, each of which corresponds to the brain wave components of a specific frequency band, such as theta waves, alpha waves, and beta waves.

[0036] Furthermore, the decomposition signals and the language template signals are combined in pairs, and the correlation between each group of decomposition signals and the corresponding language template signal is calculated using canonical correlation analysis (CCA) to obtain the maximum correlation coefficient.

[0037] Through the above steps, the filter bank canonical correlation analysis algorithm is used to perform a preliminary analysis of the steady-state visual evoked potential signal, so that the complex EEG signal can be decomposed into multiple frequency band signals that are easy to analyze. By comparing it with the preset language template signal, the feature information related to the language intention is extracted, providing basic data for subsequent signal optimization and speech conversion.

[0038] In some implementations, the multiple decomposed signals are preliminarily parsed based on the multiple language template signals according to a canonical correlation analysis algorithm, and first parsed data is output, where the expression includes: in, To decompose the signal and language template signals The degree of correlation at frequency t, For the A decomposition signal, is the language template signal at frequency t, For projecting decomposition signal , Used to project language template signals , is the decomposed signal after projection, is the language template signal after projection, T is the number of sampling points in the time series; for and The covariance of is the variance of the decomposed signal after projection, is the variance of the projected language template signal.

[0039] Specifically, in the canonical correlation analysis formula, Indicates the correlation between the decomposition signal and the language template signal at frequency t, which is used to measure the matching degree between the target decomposition signal and the corresponding language template signal; For the A decomposition signal, obtained by decomposing the filter bank, represents the EEG signal characteristics within a specific frequency range; is the language template signal at frequency t, which comes from a predefined signal library and corresponds to the language stimulus (such as image, text, etc.); A and B are projection matrices, which represent the weight vectors used for signal projection and are used to project the decomposition signal and the language template signal into a new space to better capture the correlation between them; and The high-dimensional signal is and Decomposed signal after projection mapping to one-dimensional feature space.

[0040] Specifically, is the covariance of the projected decomposition signal and the projected language template signal, and are the variances of the projected decomposition signal and the projected language template signal, respectively, which are used to describe the amplitude of signal changes.

[0041] The above-mentioned typical correlation analysis algorithm reduces the dimensionality of high-dimensional signals through projection, thereby improving computational efficiency and result interpretability. It can also perform frequency band analysis on the matching degree between multiple decomposed signals and language template signals, thereby helping to ensure the accuracy of signal analysis.

[0042] S300: Collecting facial expression sensing data and head movement sensing data of the stroke patient through a multimodal sensing device, optimizing the first analytical data according to the facial expression sensing data and the head movement sensing data, and outputting second analytical data.

[0043] Specifically, the first parsed data is optimized by combining the facial expression sensing data and head movement sensing data collected by the multimodal sensing device. The goal is to assist in judging the patient's language intention and emotional state by considering the patient's non-verbal information during the language expression process, such as facial expressions and head movements, so as to further improve the accuracy and completeness of language analysis and make the output speech closer to the patient's true intention.

[0044] Specifically, the multimodal sensing device includes at least an image acquisition device, an acceleration sensor, a gyroscope, etc. to obtain facial image information and head movement information of stroke patients, and output them as facial expression sensing data and head movement sensing data.

[0045] Specifically, facial expression sensing data refers to data related to facial expressions collected by sensors, such as muscle activity, changes in facial feature points, etc.; head movement sensing data refers to data related to head movement collected by sensors, such as acceleration, angular velocity, etc.

[0046] In some embodiments, optimizing the first parsed data based on the facial expression sensor data and the head movement sensor data to output second parsed data comprises: According to the facial expression sensing data and the head movement sensing data, a facial expression feature vector and a head movement feature vector are obtained; according to the facial expression feature vector and the head movement feature vector, an emotion recognition model is input, and an emotion state label is output according to the emotion recognition model; according to the emotion state label, the first parsed data is optimized, and second parsed data is output.

[0047] Specifically, the facial expression feature vector and the head movement feature vector are mathematical vector representations of the facial expression sensing data and the head movement sensing data, and are used as input data for the subsequent emotion recognition model; the emotion recognition model is an analysis and recognition model based on machine learning or deep learning, and is used to output an emotional state label based on the input feature vector, where the emotional state label refers to the emotional category output by the model, such as happiness, sadness, anger, etc.

[0048] Specifically, the first parsed data is optimized based on the facial expression sensor data and the head movement sensor data to output second parsed data. First, feature extraction is performed based on the facial expression sensor data and the head movement sensor data. This includes extracting features such as eyebrow movement, lip opening and closing, and facial micro-expressions. The rotation angle and posture change characteristics of head movement (such as pitch, yaw, and roll angles) are analyzed to obtain facial expression feature vectors and head movement feature vectors. Next, the facial expression feature vectors and head movement feature vectors are input into an emotion recognition model. The emotion recognition model predicts and outputs the emotional state labels corresponding to the feature vectors based on trained parameters. Finally, the first parsed data is optimized based on the emotional state labels to output second parsed data.

[0049] Exemplarily, optimizing the first parsed data according to the emotional state label includes associating and storing the emotional state label with the first parsed data, thereby introducing emotion-related information.

[0050] Specifically, to build an emotion recognition model, we first collect emotion-related multimodal sample data for training and testing, including facial expression data, head movement data, and corresponding emotion labels; then, use facial key point detection (such as OpenCV, Dlib, and other algorithms) to extract the coordinates of facial key points, and perform expression feature extraction (such as shape features, texture features, dynamic features, etc.), time-series the head movement angles (such as pitch angle, yaw angle, and roll angle), and extract corresponding head movement features (such as angle change features, movement speed and acceleration, etc.); then, select a suitable machine learning or deep learning model to build the basic framework of the emotion recognition model, and use the multimodal sample data collected above as input and emotion labels as output for supervised learning, and use cross-validation methods to verify the model to ensure the generalization ability of the model until the model performance meets expectations.

[0051] By building an emotion recognition model, the emotional state of stroke patients can be analyzed more accurately, and combined with EEG signal analysis, the accuracy of language recovery assessment in stroke patients can be further improved.

[0052] By combining emotional state labels with EEG signal analysis, emotional factors can be taken into account in the language recovery assessment of stroke patients, thereby obtaining more accurate secondary analysis data, which will help to better understand the relationship between patients' emotions and language recovery, and at the same time provide a more personalized and scientific basis for subsequent language training and rehabilitation programs.

[0053] S400: Inputting the second parsed data into a voice output module for voice conversion and output.

[0054] Specifically, the optimized second parsed data is input into the speech output module for speech conversion and output, thereby converting the patient's brain intention into audible speech, helping stroke patients overcome language barriers, communicate and interact effectively, and improve their quality of life and social participation.

[0055] Specifically, the speech output module is a key component that converts the second parsed data into audible speech. This module generates speech content that matches the EEG activity based on the patient's EEG signals and emotional state, and adjusts the emotional color of the speech according to the emotional state label (such as anxiety, relaxation, happiness, etc.). For example, the speaking speed is faster when anxious, and the tone is steady when relaxed. Exemplarily, the speech output module uses text-to-speech (TTS) synthesis technology to convert the generated text into speech.

[0056] In some embodiments, after inputting the second parsed data into a speech output module for speech conversion and output, the method further includes: A rehabilitation training module is set up, which is used to store the second analytical data and obtain an analytical data sample set; input the analytical data sample set into a rehabilitation assessment model, and output rehabilitation assessment indicators according to the rehabilitation assessment model; connect to a rehabilitation training device, send the rehabilitation assessment indicators to the rehabilitation training device for rehabilitation training project feedback optimization, and output the optimized rehabilitation training project, wherein the rehabilitation training project feedback optimization includes training intensity and training frequency.

[0057] Specifically, the rehabilitation training module is used to store and manage the patient's rehabilitation training data; the rehabilitation assessment model is an algorithm or system used to output rehabilitation assessment indicators, such as rehabilitation progress, training intensity and training frequency, based on the parsed data sample set; the parsed data sample set is a set of samples collected from the second parsed data, used to evaluate the patient's rehabilitation progress; the rehabilitation training device is a device used to implement rehabilitation training, such as virtual reality (VR) equipment, physical therapy equipment, etc.

[0058] Specifically, the parsed data sample set is input into the rehabilitation assessment model, and rehabilitation assessment indicators are output according to the rehabilitation assessment model, including rehabilitation progress, training intensity and training frequency. For example, if the patient's language expression ability is weak, the training intensity can be appropriately increased; if the patient has shown a good recovery state, the training intensity can be appropriately reduced. For patients with slower emotional recovery, more training frequency may be required; and for patients with better recovery, the training frequency can be reduced; then, the rehabilitation training device is connected, and the rehabilitation assessment indicators are sent to the rehabilitation training device for feedback optimization of the rehabilitation training program, and the frequency, intensity and time of the training parameters are adaptively adjusted to output the optimized rehabilitation training program.

[0059] The above steps achieve accurate tracking and dynamic adjustment of the language ability and emotional recovery of stroke patients by integrating the speech output module, rehabilitation training module and evaluation model, thereby providing personalized rehabilitation training optimization and improving the patient's rehabilitation effect.

[0060] In some embodiments, the second parsed data is input into a speech output module for speech conversion and output, and the method further includes: Acquire the speech conversion output result; perform speech accuracy recognition on the speech conversion output result and output an accuracy index. If the accuracy index is less than a preset accuracy, activate a sentence correction module, and the sentence correction module is used to perform sentence correction output on the second parsed data.

[0061] Specifically, in order to evaluate the quality and accuracy of speech output, speech recognition technology is used to analyze the converted speech and calculate the speech accuracy, where the accuracy can be determined by comparing the actual speech with the expected language template or standard speech. For example, the speech recognition system evaluates the correctness of words, sentences, and emotional expressions in the speech and outputs an accuracy index, which represents the performance of the patient's speech output in the speech recognition system, reflecting the clarity, expressiveness and match of the speech output with the expected content.

[0062] Specifically, if the speech accuracy is less than the preset accuracy threshold, indicating that there are errors in the patient's speech output or difficulty in understanding, the sentence correction module is activated to correct the patient's speech output, including correcting grammatical errors, pronunciation errors or inappropriate emotional expressions in the speech, and optimizing the speech output through emotion recognition technology to make it more in line with the patient's emotional state, ensuring the accuracy and naturalness of the speech output, etc.; specifically, if the speech accuracy meets the standard, the correction process can be skipped.

[0063] The above steps, through the collaborative work of speech output, accuracy recognition, and sentence correction modules, can optimize the language expression of stroke patients in real time, ensuring that patients can output speech clearly and accurately based on EEG signal analysis.

[0064] In summary, the language analysis and assessment method for stroke patients provided by the present invention has the following technical effects: The EEG signals of stroke patients, including steady-state visual evoked potential signals, are acquired through non-invasive EEG signal acquisition equipment; the acquired steady-state visual evoked potential signals are preliminarily analyzed through the filter group canonical correlation analysis algorithm to generate first analysis data; the facial expression sensor data and head movement sensor data of the patient are acquired using a multimodal sensing device, and the first analysis data are optimized based on these data to obtain second analysis data; the optimized second analysis data is input into the speech output module to complete the speech conversion and output, thereby achieving the technical effect of improving the accuracy of the assessment of the patient's expression intention and improving the patient's communication ability and willingness to participate.

[0065] Example 2 Figure 2 This is a schematic diagram of the structure of a language analysis and evaluation system for stroke patients according to the present invention. For example, Figure 1 The flow chart of the language analysis and evaluation method for stroke patients in the present invention can be shown as follows: Figure 2 The structure shown is implemented.

[0066] Based on the same concept as the language analysis and evaluation method for stroke patients in the above embodiment, the present invention further provides a language analysis and evaluation system for stroke patients, comprising: The EEG signal acquisition module 11 is used to acquire EEG signals from stroke patients using non-invasive EEG signal acquisition equipment, where the EEG signals include steady-state visual evoked potential signals.

[0067] The preliminary analysis module 12 is used to perform preliminary analysis on the steady-state visual evoked potential signal by using a filter bank canonical correlation analysis algorithm and output first analysis data.

[0068] The multimodal data optimization module 13 is used to collect facial expression sensor data and head movement sensor data of the stroke patient through a multimodal sensing device, optimize the first parsed data according to the facial expression sensor data and the head movement sensor data, and output second parsed data.

[0069] The voice output module 14 is configured to input the second parsed data into the voice output module for voice conversion and output.

[0070] In some embodiments, the EEG signal further includes an Alpha wave signal, and the EEG signal acquisition module 11 includes an acquisition and discrimination unit for: The signal strength of the Alpha wave signal is extracted based on frequency domain analysis; a signal strength threshold is set, and the Alpha wave signal is judged according to the signal strength threshold; if the signal strength of the Alpha wave signal is greater than or equal to the signal strength threshold, the steady-state visual evoked potential signal is collected; if the signal strength of the Alpha wave signal is less than the signal strength threshold, the collection of the steady-state visual evoked potential signal is stopped, and the non-invasive EEG signal collection device is placed in a standby state.

[0071] In some implementations, the execution steps of the acquisition and discrimination unit also include: obtaining the Alpha wave sample signal of the stroke patient; performing a signal intensity change sensitivity analysis on the Alpha wave sample signal, outputting a sensitivity index, and setting the signal intensity threshold of the stroke patient according to the sensitivity index.

[0072] In some embodiments, in the preliminary analysis module 12, the filter bank includes a plurality of band-pass filters, and the steady-state visual evoked potential signal is subjected to multi-band decomposition according to the plurality of band-pass filters to extract a plurality of decomposed signals. The preliminary analysis module 12 includes: The language template signal acquisition unit is configured to acquire a plurality of language template signals, where the plurality of language template signals correspond to the plurality of decomposition signals.

[0073] The preliminary parsing unit is configured to perform preliminary parsing on the multiple decomposition signals based on the multiple language template signals according to a canonical correlation analysis algorithm, and output first parsed data.

[0074] In some implementations, the multiple decomposed signals are preliminarily parsed based on the multiple language template signals according to a canonical correlation analysis algorithm, and first parsed data is output, where the expression includes: in, To decompose the signal and language template signals The degree of correlation at frequency t, For the A decomposition signal, is the language template signal at frequency t, For projecting decomposition signal , Used to project language template signals , is the decomposed signal after projection, is the language template signal after projection, T is the number of sampling points in the time series; for and The covariance of is the variance of the decomposed signal after projection, is the variance of the projected language template signal.

[0075] In some embodiments, the multimodal data optimization module 13 includes: The feature vector acquisition unit is used to acquire a facial expression feature vector and a head movement feature vector according to the facial expression sensing data and the head movement sensing data.

[0076] The emotional state label output unit is used to input the facial expression feature vector and the head movement feature vector into an emotion recognition model and output an emotional state label according to the emotion recognition model.

[0077] The parsed data optimization unit is configured to optimize the first parsed data according to the emotional state label and output second parsed data.

[0078] In some embodiments, the voice output module 14 further includes: The rehabilitation training module setting unit is used to set a rehabilitation training module, and the rehabilitation training module is used to store the second parsed data and obtain a parsed data sample set.

[0079] The rehabilitation assessment index output unit is used to input the analysis data sample set into the rehabilitation assessment model and output the rehabilitation assessment index according to the rehabilitation assessment model.

[0080] The rehabilitation training program feedback optimization unit is used to connect to the rehabilitation training device, send the rehabilitation assessment indicators to the rehabilitation training device for rehabilitation training program feedback optimization, and output the optimized rehabilitation training program, wherein the rehabilitation training program feedback optimization includes training intensity and training frequency.

[0081] In some embodiments, the voice output module 14 further includes: The voice conversion output result obtaining unit is used to obtain the voice conversion output result.

[0082] The speech accuracy recognition unit is used to perform speech accuracy recognition on the speech conversion output result and output an accuracy index. If the accuracy index is less than a preset accuracy, the sentence correction module is activated. The sentence correction module is used to perform sentence correction output on the second parsed data.

[0083] Example 3 Figure 3 The schematic structural diagram of an exemplary electronic device provided for the present invention shows a block diagram of an exemplary electronic device suitable for implementing an embodiment of the present invention. Figure 3The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 31, a memory 32, an input device 33 and an output device 34; the number of processors 31 in the electronic device can be one or more. Figure 3 Taking a processor 31 as an example, the processor 31, memory 32, input device 33 and output device 34 in the electronic device can be connected through a bus or other means. Figure 3 The bus connection is taken as an example.

[0084] Memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for language parsing and assessment of stroke patients in an embodiment of the present invention. Processor 31 executes the software programs, instructions, and modules stored in memory 32 to perform various computer functions and data processing, thereby implementing the aforementioned method for language parsing and assessment of stroke patients.

[0085] It should be understood that the embodiments mentioned in this specification focus on their differences from other embodiments. The specific embodiments in the aforementioned embodiment one are also applicable to the language analysis and evaluation system for stroke patients described in embodiment two. For the sake of brevity of the specification, no further elaboration will be given here.

[0086] It should be understood that the embodiments disclosed in the present invention and the above description can enable those skilled in the art to use the present invention to implement the present invention. At the same time, the present invention is not limited to the embodiments mentioned above. It should be understood that those skilled in the art can still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention and are all included in the scope of protection of the present invention.

Claims

1. A language analysis and assessment method for stroke patients, characterized in that: The method comprises: Collecting EEG signals from stroke patients using a non-invasive EEG signal acquisition device, wherein the EEG signals include steady-state visual evoked potential signals; Performing a preliminary analysis on the steady-state visual evoked potential signal using a filter bank canonical correlation analysis algorithm, and outputting first analysis data; collecting facial expression sensor data and head movement sensor data of the stroke patient through a multimodal sensing device, optimizing the first parsed data according to the facial expression sensor data and the head movement sensor data, and outputting second parsed data; The second parsed data is input into a voice output module for voice conversion and output.

2. A method for language analysis and assessment of stroke patients according to claim 1, characterized in that: The EEG signal also includes an Alpha wave signal; Extracting the signal strength of the Alpha wave signal based on frequency domain analysis; setting a signal strength threshold, judging the Alpha wave signal according to the signal strength threshold, and acquiring the steady-state visual evoked potential signal if the signal strength of the Alpha wave signal is greater than or equal to the signal strength threshold; If the signal strength of the Alpha wave signal is less than the signal strength threshold, the acquisition of the steady-state visual evoked potential signal is stopped, and the non-invasive EEG signal acquisition device is placed in a standby state.

3. A method for language analysis and assessment of stroke patients as claimed in claim 2, characterized in that: The method for setting the signal strength threshold includes: Acquiring an Alpha wave sample signal from the stroke patient; Perform signal intensity change sensitivity analysis on the Alpha wave sample signal, output a sensitivity index, and set a signal intensity threshold for the stroke patient based on the sensitivity index.

4. A method for language analysis and assessment of stroke patients according to claim 1, characterized in that: After inputting the second parsed data into a voice output module for voice conversion and output, the method further includes: Setting a rehabilitation training module, the rehabilitation training module is used to store the second parsed data and obtain a parsed data sample set; Inputting the analyzed data sample set into a rehabilitation assessment model, and outputting rehabilitation assessment indicators according to the rehabilitation assessment model; Connecting to a rehabilitation training device, sending the rehabilitation assessment index to the rehabilitation training device for rehabilitation training item feedback optimization, and outputting the optimized rehabilitation training item, wherein the rehabilitation training item feedback optimization includes training intensity and training frequency.

5. A method for language analysis and assessment of stroke patients according to claim 4, characterized in that: Inputting the second parsed data into a voice output module for voice conversion and output, the method further includes: Get the speech conversion output result; The speech conversion output result is subjected to speech accuracy recognition and an accuracy index is output. If the accuracy index is less than a preset accuracy, a sentence correction module is activated. The sentence correction module is used to perform sentence correction output on the second parsed data.

6. A method for language analysis and assessment of stroke patients according to claim 1, characterized in that: The steady-state visual evoked potential signal is preliminarily analyzed by a filter bank canonical correlation analysis algorithm to output first analysis data. include: The filter bank includes a plurality of band-pass filters, and the steady-state visual evoked potential signal is subjected to multi-band decomposition according to the plurality of band-pass filters to extract a plurality of decomposed signals; Acquire a plurality of language template signals, where the plurality of language template signals correspond to the plurality of decomposed signals; The multiple decomposition signals are preliminarily analyzed based on the multiple language template signals according to a canonical correlation analysis algorithm, and first analysis data is output.

7. A method for language analysis and assessment of stroke patients according to claim 6, characterized in that: The multiple decomposition signals are preliminarily parsed based on the multiple language template signals according to a canonical correlation analysis algorithm, and first parsed data is output. The expression includes: in, To decompose the signal and language template signals The degree of correlation at frequency t, For the A decomposition signal, is the language template signal at frequency t, For projecting decomposition signal , Used to project language template signals , is the decomposed signal after projection, is the language template signal after projection, T is the number of sampling points in the time series; for and The covariance of is the variance of the decomposed signal after projection, is the variance of the projected language template signal.

8. A method for language analysis and assessment of stroke patients according to claim 1, characterized in that: Optimizing the first parsed data according to the facial expression sensor data and the head movement sensor data to output second parsed data, the method comprising: Acquire a facial expression feature vector and a head movement feature vector according to the facial expression sensing data and the head movement sensing data; Inputting the facial expression feature vector and the head movement feature vector into an emotion recognition model, and outputting an emotional state label according to the emotion recognition model; The first parsed data is optimized according to the emotional state label, and second parsed data is output.

9. A language analysis and assessment system for stroke patients, characterized in that: The system is used to execute the method for language analysis and assessment of stroke patients according to any one of claims 1 to 8, and the system comprises: An EEG signal acquisition module is used to acquire EEG signals from stroke patients using a non-invasive EEG signal acquisition device, wherein the EEG signals include steady-state visual evoked potential signals; a preliminary analysis module, configured to perform preliminary analysis on the steady-state visual evoked potential signal using a filter bank canonical correlation analysis algorithm and output first analysis data; a multimodal data optimization module, configured to collect facial expression sensing data and head movement sensing data of the stroke patient through a multimodal sensing device, optimize the first parsed data according to the facial expression sensing data and the head movement sensing data, and output second parsed data; The voice output module is used to input the second parsed data into the voice output module for voice conversion and output.

10. An electronic device, characterized in that: The electronic device comprises: a memory for storing executable instructions; The processor is configured to implement the language analysis and assessment method for stroke patients according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

Images