Concentration training method and system based on multi-modal fusion

Abstract

The invention discloses a concentration training method and system based on multi-modal fusion, and the method comprises the steps: collecting an electroencephalogram signal, an eye movement signal and a head movement signal through a wearable device in a concentration training process, and calculating an electroencephalogram frequency characteristic parameter, an eye movement parameter and a head movement modal parameter in a one-to-one correspondence manner; calculating a concentration score according to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters so as to evaluate the concentration; and adjusting a concentration training strategy according to a concentration evaluation result. According to the method, the electroencephalogram, eye movement and head movement signals are collected at the same time through the wearable device, the corresponding characteristic parameters are calculated, the concentration score is comprehensively evaluated, and a comprehensive, accurate and stable evaluation result is provided. On the basis of real-time evaluation, task difficulty and types are dynamically adjusted, challenging and interestingness are ensured, the user participation degree and the training effect are improved, and concentration research and technical progress are promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of concentration training, and specifically relates to a concentration training method and system based on multimodal fusion. Background Art

[0002] In today's digital information age, people's attention is constantly disturbed by various factors, and decreased concentration has become a common problem. Traditional concentration training methods often rely on static exercises or paper-and-pen tasks, which are not interesting and have limited effects, and are difficult to adapt to the fast-paced lifestyle and personalized needs of modern society.

[0003] With the continuous development of sensor technology, signal processing technology and mobile application development technology, integrating a variety of biodetection technologies into smart wearable devices and combining them with highly interactive applications (APPs) has made it possible to develop efficient and interesting concentration training tools.

[0004] In the process of concentration training, one modal parameter is generally used to evaluate concentration. However, the data source obtained by only one mode is limited and cannot fully reflect the multi-dimensional physiological and behavioral characteristics of the user. At the same time, a single mode is also easily affected by external factors, which may lead to high volatility in the evaluation results. In addition, the evaluation criteria set based on a single mode are usually fixed and cannot adapt well to the unique situation of each user. For example, some people are born with slow eye movement reactions, and when only eye movement signals are used for concentration evaluation, large errors may occur.

[0005] In addition, since the concentration training tasks are pre-set and there is a lack of a feedback mechanism for the concentration assessment results, during a single concentration training task, the preset tasks cannot be dynamically adjusted according to the user's actual performance. This may cause the task to be too simple and unchallenging, or too difficult to make the user feel frustrated. Training based on preset tasks often adopts a unified standard, ignoring individual differences, resulting in a mismatch between training content and personal needs, which greatly affects the user's training enthusiasm and concentration training effect. Summary of the invention

[0006] The present invention provides a concentration training method and system based on multimodal fusion to solve the problems of large errors in evaluation results caused by using one modal parameter to evaluate concentration during concentration training, as well as poor concentration training effect caused by lack of a feedback mechanism for concentration evaluation results and inability to dynamically adjust preset training tasks.

[0007] The technical solution adopted by the present invention is:

[0008] A concentration training method based on multimodal fusion is applied to a concentration training system including wearable devices, including:

[0009] During the concentration training, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one;

[0010] Calculating a concentration score according to the EEG frequency characteristic parameter, the eye movement parameter, and the head movement modality parameter to evaluate concentration;

[0011] Adjust your concentration training strategy based on the concentration assessment results.

[0012] The concentration score is calculated according to the EEG frequency characteristic parameter, the eye movement parameter and the head movement modality parameter, specifically:

[0013] Setting a first regression parameter, a second regression parameter and a third regression parameter in one-to-one correspondence with the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters, and linearly combining and superimposing the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters to obtain the concentration score;

[0014] The first regression parameter, the second regression parameter and the third regression parameter are obtained by fitting and calculating based on the sample data marked with the concentration state.

[0015] Adjust the concentration training strategy based on the results of the concentration assessment, specifically:

[0016] Adjust the difficulty of training tasks based on the results of the concentration assessment;

[0017] and / or, providing rewards or penalties for training tasks based on the results of the concentration assessment;

[0018] And / or, based on the results of the concentration assessment, complete training tasks to advance to the next level.

[0019] During the concentration training, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one, specifically:

[0020] The EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, the eye movement parameter is calculated based on the stabilization time of the eye gaze point, and the head movement modal parameter is calculated based on the head stillness time.

[0021] The EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, specifically:

[0022] The electroencephalogram signal includes alpha wave signal, beta wave signal and theta wave signal.

[0023] The EEG frequency characteristic parameter is the ratio of the α wave power spectrum density to the EEG signal power spectrum density, or,

[0024] The EEG frequency characteristic parameter is the ratio of the β wave power spectrum density to the EEG signal power spectrum density, or,

[0025] The EEG frequency characteristic parameter is the ratio of the θ wave power spectrum density to the EEG signal power spectrum density, or,

[0026] The value of the EEG frequency characteristic parameter is the ratio of the α wave power spectral density to the θ wave power spectral density.

[0027] The eye movement parameters are calculated based on the eye gaze point stabilization time, specifically,

[0028] The eye movement parameter is the ratio of the eye gaze point stability time to the total observation time.

[0029] Wherein, when the average fixation time on a specific visual target or area is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or,

[0030] When the saccade speed between different fixations is between 300-600° / s, and / or,

[0031] When the blinking frequency is between 10-20 times / minute,

[0032] The eye gaze point is judged to be stable.

[0033] The head motion modal parameters are calculated based on the head static time, specifically:

[0034] The head motion modal parameter is the ratio of the head static time to the total observation time.

[0035] Among them, when the average acceleration of the head movement is less than 0.2m / s 2 When the average angular velocity amplitude is less than 5° / s, the head is judged to be stationary.

[0036] The method further comprises,

[0037] The concentration assessment results are displayed in real time on the concentration training interface to remind the concentration status during the concentration training process in real time;

[0038] When concentration decreases, voice or image prompts are output through the wearable device.

[0039] The present invention further provides a concentration training system, including a wearable device, wherein the wearable device is smart glasses, wherein the smart glasses include lenses, a frame for fixing the lenses, a nose pad and temples, the nose pads are provided with electrodes for collecting brain electrical signals, the frame is provided with a camera for collecting eye movement signals, the frame or the temples are provided with an inertial measurement unit for collecting head movement signals, and the frame or the temples are provided with a processing unit for processing the brain electrical signals, the eye movement signals and the head movement signals, so as to adjust the concentration training strategy according to the processing results.

[0040] The concentration training system also includes an application used in conjunction with the wearable device, wherein a plurality of training game modes are arranged in the application. The application can receive the concentration evaluation results obtained by the wearable device and form a training data display interface.

[0041] Due to the adoption of the above technical solution, the beneficial effects achieved by the present invention are as follows:

[0042] 1. In the present invention, during the concentration training process, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one; according to the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters, the concentration score is calculated to evaluate the concentration. By simultaneously collecting three different types of signals, namely EEG, eye movement and head movement, the physiological and behavioral state of the user can be more comprehensively reflected, avoiding the limitations of a single modality data source and improving the accuracy of concentration assessment. In addition, multiple signals complement each other, reducing the risk of misjudgment caused by the influence of the environment or individual differences on a single modality, making the evaluation results more stable and reliable.

[0043] 2. In the present invention, during the training process, data from multiple sensors can be processed and analyzed in real time, concentration scores can be quickly given, and timely feedback information can be provided for concentration training. Not only can the user's training data be tracked for a long time and a personal learning profile be established, so as to continuously optimize the training path and provide more targeted suggestions and support. Moreover, the specific situation of each user (such as physiological characteristics, psychological state, etc.) can be taken into account, and the system can flexibly adjust the evaluation criteria and training content to better meet the needs of different people. At the same time, the rich multimodal data provides researchers with valuable research materials, which helps to deeply understand the factors and mechanisms affecting human concentration, thereby promoting technological progress in the field of concentration research and formulating more efficient concentration training methods.

[0044] 3. In the present invention, the concentration training strategy is adjusted according to the concentration evaluation results. Based on the real-time evaluation results, the system can dynamically adjust the difficulty and type of training tasks to ensure that each training can maintain appropriate challenges and fun, and avoid users feeling bored or frustrated. The diversified training task design makes the whole process no longer monotonous and boring, improves the user's willingness and persistence to participate, and thus improves the effectiveness of concentration training. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0046] Figure 1 A schematic diagram of a flow chart of the concentration training method based on multimodal fusion in one embodiment of the present invention;

[0047] Figure 2 This is a schematic structural diagram of the smart glasses according to an embodiment of the present invention;

[0048] Figure 3 This is an operation logic diagram of the concentration training system according to one embodiment of the present invention.

[0049] Among them, 1 is smart glasses; 11 is lenses; 12 is frames; 12 is cameras; 13 is nose pads; 13 is electrodes; 14 is temples; 15 is inertial measurement unit; 16 is processing unit; 17 is Bluetooth communication module; 18 is power management module; and 19 is bone conduction earphone module. DETAILED DESCRIPTION

[0050] In order to more clearly illustrate the overall concept of the present invention, a detailed description is given below in an exemplary manner in conjunction with the accompanying drawings.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0052] like Figure 1 As shown, a concentration training method based on multimodal fusion is applied to a concentration training system including a wearable device, comprising:

[0053] S100: During the concentration training, the wearable device collects brain wave signals, eye movement signals and head movement signals, and calculates brain wave frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one.

[0054] It should be noted that the wearable device in this method is a head-mounted device, so as to integrate the functions of collecting EEG signals, eye movement signals and head movement signals. Specifically, the wearable device is a smart glasses 1, and the augmented reality (AR), virtual reality (VR) and other functions of the smart glasses 1 are used to facilitate the design and simulation of training tasks.

[0055] The smart glasses 1 are provided with electrodes 131, which are flexible, highly conductive dry electrode arrays that can accurately capture EEG signals from different areas of the brain. The smart glasses 1 are also provided with a micro camera, which uses an ultra-small, high-resolution MEMS camera and can collect eye movement video images during training. The smart glasses 1 are also provided with an inertial measurement unit 15 (IMU), which uses a high-precision, low-power MEMS-IMU chip, integrates a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer, and can collect head movement signals during training.

[0056] It is understandable that the electroencephalogram (EEG) can capture changes in electrical signals in the cerebral cortex, which are closely related to cognitive processes. Brain waves of different frequency bands (such as delta, theta, alpha, beta, gamma) correspond to different psychological states and cognitive functions, such as relaxation, concentration, alertness, etc. A large number of studies have shown that specific EEG characteristics (such as the stability of alpha waves and the activity of beta waves) can directly characterize an individual's attention level. Therefore, EEG is one of the important physiological indicators for assessing concentration.

[0057] Eyes are the main organ for humans to obtain external information, and eye movement signals can reveal the degree of attention users pay to specific targets. By recording parameters such as the dwell time on the gaze point, the length of the scanning path, and the blinking frequency, the user's visual concentration can be effectively measured. Compared with internal psychological states, eye movement behavior is more intuitive and easy to measure, providing direct evidence of attention allocation. In addition, it can also help identify which external factors may interfere with the user's concentration.

[0058] Head movement reflects an individual's ability to control their body posture, and a stable head position is often associated with higher levels of concentration. Frequent or violent head shaking may be a sign of distraction or anxiety. When a person is highly focused, they tend to remain relatively still to reduce unnecessary movement distractions. Therefore, monitoring head movement can help further verify the level of concentration indicated by the other two signals.

[0059] The present invention can more comprehensively reflect the user's physiological and behavioral state by simultaneously collecting three different types of signals, namely, EEG, eye movement, and head movement, avoiding the limitations of a single modality data source and improving the accuracy of concentration assessment. In addition, multiple signals complement each other, reducing the risk of misjudgment caused by the influence of the environment or individual differences on a single modality, making the assessment results more stable and reliable.

[0060] EEG signal collection, eye movement signal collection and head movement signal collection are more convenient for integration on wearable devices, and are convenient for the industrial layout and design of wearable devices. Of course, multiple features such as heart rate variability (HRV), galvanic skin response (GSR), breathing rate, facial expression analysis, training task completion efficiency, etc. can also be used to evaluate concentration, and the present invention does not limit this.

[0061] S200: Calculating a concentration score according to the EEG frequency characteristic parameters, the eye movement parameters and the head movement modality parameters to evaluate concentration.

[0062] The above three types of feature parameters are integrated to form a multi-dimensional data set. Each feature parameter is assigned a different weight, and the specific weight can be determined based on experimental results or expert knowledge. Use machine learning or deep learning algorithms to build a comprehensive evaluation model. For example, linear regression models, support vector machines (SVMs), random forests, or neural networks can be used. The goal of the model is to predict the user's concentration score based on the input multimodal feature parameters. It should be noted that during the model setting process, a large amount of sample data labeled with concentration status is collected for training and verifying the evaluation model. The model parameters are optimized through methods such as cross-validation to ensure its generalization ability and accuracy.

[0063] The multimodal feature parameters collected in real time are input into the trained evaluation model, and a concentration score between 0 and 100 is output, where 100 represents the highest concentration and 0 represents complete distraction. Based on the concentration score calculated in real time, the system can immediately provide feedback to the user on their current concentration status.

[0064] S300: Adjust the concentration training strategy based on the concentration assessment results.

[0065] In this method, the user's current state of concentration can be immediately fed back, and the difficulty and other parameters of the training task can be adjusted accordingly. If the user shows high concentration, the task complexity or speed can be increased to encourage the user to continue to maintain a high level of concentration; conversely, if the concentration decreases, the difficulty can be appropriately reduced and more prompts and support can be provided.

[0066] Alternatively, design a series of tasks from easy to difficult, gradually introducing more challenging content as the user's concentration increases, maintaining an appropriate level of challenge without frustrating the user.

[0067] In summary, through real-time feedback and dynamic adjustment, the training strategy can be flexibly adjusted according to the results of concentration assessment, so as to achieve more efficient and personalized concentration training. This method not only helps to improve the user's concentration level, but also significantly improves their performance in daily life and work.

[0068] As a preferred embodiment of the present invention, the concentration score is calculated according to the EEG frequency characteristic parameters, the eye movement parameters and the head movement modality parameters, specifically:

[0069] Setting a first regression parameter, a second regression parameter and a third regression parameter in one-to-one correspondence with the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters, and linearly combining and superimposing the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters to obtain the concentration score;

[0070] The first regression parameter, the second regression parameter and the third regression parameter are obtained by fitting and calculating based on the sample data marked with the concentration state.

[0071] The concentration evaluation model built based on big data analysis and machine learning algorithms comprehensively considers multimodal information such as EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters to conduct real-time quantitative evaluation of the user's concentration level, with the goal of minimizing the error between the predicted value and the actual value. The model constructs a mapping relationship between EEG frequency characteristic parameters, eye movement parameters, head movement modal parameters, and concentration scores, and can predict the user's concentration state based on new input data (i.e., user data collected in real time).

[0072] Specifically, the concentration score is approximately represented by a linear combination of multiple factors such as EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters. This model is relatively simple, easy to understand and implement, and is suitable for preliminary concentration assessment.

[0073] The model expression is:

[0074] y=B0+B1X1+B2X2+B3X3+z

[0075] Where y is the concentration score (the value range can be 0-100, 0 means completely distracted, 100 means highly focused);

[0076] X1 is the EEG frequency characteristic parameter; X2 is the eye movement parameter; X3 is the head movement modal parameter;

[0077] B0 is the intercept, which represents the basic score of concentration when all independent variables are 0;

[0078] B1 is the first regression parameter; B2 is the second regression parameter; B3 is the third regression parameter, which indicates the degree of influence of each independent variable on the concentration score;

[0079] z is the error term, which is used to take into account other factors not included in the model and measurement errors.

[0080] By collecting a large amount of sample data with annotated concentration states (for example, by having the subjects simultaneously collect brain wave, head movement and eye movement data in tasks with different attention requirements, and manually assessing their concentration states), the values ​​of B0, B1, B2 and B3 are estimated using methods such as the least squares method. For example, after fitting the training data, B0 = 30, B1 = 20, B2 = 30, and B3 = 20 may be obtained.

[0081] As a preferred embodiment of the present invention, the concentration training strategy is adjusted according to the concentration evaluation result, specifically:

[0082] Adjust the difficulty of training tasks based on the results of the concentration assessment;

[0083] and / or, providing rewards or penalties for training tasks based on the results of the concentration assessment;

[0084] And / or, based on the results of the concentration assessment, complete training tasks to advance to the next level.

[0085] In this embodiment, the user's concentration score is continuously monitored, and the brain wave, eye movement and head movement signals are collected in real time through the smart wearable device to calculate the current concentration level. According to the change of concentration score, the system automatically adjusts the difficulty of the training task. The concentration evaluation results can be divided into:

[0086] High concentration (score above 80): Increase the complexity or speed of the task, such as speeding up the target movement in a visual tracking game, increasing the number of targets, or changing the complexity of the trajectory.

[0087] Moderate Focus (score 50-79): Keep the difficulty of existing tasks unchanged, but introduce some new elements to maintain the challenge.

[0088] Low concentration (score below 50): Reduce the difficulty of the task, simplify the operation steps or reduce distractions, provide more prompts and support to help users refocus.

[0089] The present invention does not limit the concentration training strategy, and any one of the following embodiments may be adopted, or the following embodiments may be used in any combination.

[0090] Embodiment 1: Adjust the difficulty of training tasks according to the results of concentration assessment. Design a variety of game tasks based on concentration training. For example, design a visual tracking game in which multiple moving targets appear on the screen, requiring players to track specific targets with their eyes and keep their gaze. The game is initially set to a lower difficulty, such as slower moving targets and fewer targets. As the game progresses, the difficulty is dynamically adjusted according to the results of the concentration assessment. If the player maintains high concentration for a certain period of time, increase the number of targets, increase the moving speed, or change the complexity of the moving trajectory; conversely, if the player performs poorly, reduce the difficulty appropriately to give the player more successful experiences to enhance confidence and training effects.

[0091] Embodiment 2: Reward or punish training tasks based on the results of the concentration assessment. Provide users with a points system, and when they successfully complete a specific attention task within the specified time, they can obtain virtual points or unlock new game scenes. When the user performs well, the system can give positive feedback through voice praise, animation effects, etc., to enhance the user's sense of accomplishment and enthusiasm. At the same time, set up weekly / monthly rankings to encourage users to compete for higher rankings; you can also set milestone achievement badges to record the user's progress.

[0092] If the user fails the task due to distraction (such as losing focus or clicking the wrong button), a certain number of points will be deducted or a short "penalty time" (such as pausing the game for a few seconds) will be given to encourage the user to concentrate more. At the same time, the user is allowed to quickly recover the deducted points or shorten the penalty time through extra efforts (such as completing simple auxiliary tasks) to avoid excessively undermining the user's confidence.

[0093] Example 3: According to the results of the concentration assessment, the training tasks are passed and advanced. The entire training process is divided into multiple levels or stages, each with different task requirements and increasing difficulty. Users need to reach certain completion standards (such as accumulating enough points or completing a specific number of tasks) at the current level before they can enter the next level. Different types of tasks are designed for different levels to comprehensively train users' attention in all aspects, such as sustained attention, selective attention, distributed attention, etc.

[0094] A comprehensive concentration score is generated based on the user's performance in each training task (such as average gaze accuracy, reaction time distribution, etc.). Only when the score reaches the preset threshold can the user successfully advance to the next level. For levels that fail to pass, the system will analyze the user's performance data and give targeted improvement suggestions to help users identify and improve weak links, thereby increasing the success rate of the next attempt.

[0095] This implementation method can flexibly adjust the training strategy according to the concentration assessment results by dynamically adjusting the task difficulty, implementing a reward or punishment mechanism, and performing task clearance and advancement, thereby achieving more efficient and personalized concentration training and helping to improve the user's concentration level.

[0096] As a preferred embodiment of the present invention, during the concentration training, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one, specifically:

[0097] The EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, the eye movement parameter is calculated based on the stabilization time of the eye gaze point, and the head movement modal parameter is calculated based on the head stillness time.

[0098] It is understandable that the EEG module on the smart wearable device can capture the EEG signals generated by brain activity. These signals cover multiple frequency bands (δ, θ, α, β, γ), each of which corresponds to a different cognitive state.

[0099] Delta waves (0.5-4Hz), commonly seen during deep sleep or extreme relaxation;

[0100] Theta waves (4-8Hz), which occur during mild relaxation or lack of concentration;

[0101] Alpha waves (8-13Hz), which are more obvious in a relaxed but focused state;

[0102] Beta waves (13-30Hz), active when alert, thinking, or problem solving;

[0103] Gamma waves (30-100Hz) are high-frequency activities during complex cognitive tasks.

[0104] For each frequency band, calculate its power spectral density (PSD), that is, the energy distribution of the signal in the frequency band. This can be achieved through mathematical tools such as fast Fourier transform (FFT).

[0105] Calculating the ratio between specific frequency bands, such as the ratio of alpha waves to theta waves (Palpha / Ptheta), is used to assess the degree of relaxation and concentration; or the proportion of beta waves, as an indicator of high alertness or tension.

[0106] The final output is the PSD value of each frequency band and its proportional relationship. These values ​​reflect the user's current cognitive and emotional state and are one of the important physiological indicators for evaluating concentration.

[0107] In addition, the camera 121 on the wearable device is used to record the position and movement trajectory of the eyes, including the dwell time on the gaze point, the length of the scanning path, and the blinking frequency.

[0108] Identify the user's gaze behavior, that is, the period of time when the eyes remain fixed on a fixed target. Calculate the duration of each gaze and calculate the average gaze time over a period of time. Long and stable gazes generally indicate higher concentration, while frequent and short gazes may indicate distraction.

[0109] Again, an inertial measurement unit 15 (IMU) on the wearable device, such as an accelerometer, gyroscope, and magnetometer, is used to monitor changes in head posture, including acceleration and angular velocity amplitude.

[0110] Monitor the movement of the head in real time and identify the time periods when the head is relatively still. Calculate the proportion of time the head remains still in the total observation time. Longer still time is usually associated with higher concentration, because a stable head posture helps reduce external distractions and allows users to focus more on the current task.

[0111] By calculating the above three characteristic parameters, the system can comprehensively capture the user's physiological and behavioral characteristics, thereby achieving accurate quantitative assessment of the concentration state.

[0112] As an example of this implementation, the EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, specifically:

[0113] The electroencephalogram signal includes alpha wave signal, beta wave signal and theta wave signal.

[0114] The EEG frequency characteristic parameter is the ratio of the α wave power spectrum density to the EEG signal power spectrum density, or,

[0115] The EEG frequency characteristic parameter is the ratio of the β wave power spectrum density to the EEG signal power spectrum density, or,

[0116] The EEG frequency characteristic parameter is the ratio of the θ wave power spectrum density to the EEG signal power spectrum density, or,

[0117] The value of the EEG frequency characteristic parameter is the ratio of the α wave power spectral density to the θ wave power spectral density.

[0118] The EEG frequency characteristic parameter is the ratio of the α wave power spectrum density to the EEG signal power spectrum density. The ratio of the α wave power spectrum density (corresponding value output by the EEG module) to the total EEG signal power spectrum density (Pa / Ptotal) is calculated.

[0119] When the ratio is greater than 0.2 and the fluctuation range is less than 0.05 within a period of time (such as 30 seconds), it can be considered that the alpha wave is in a relatively stable and strong state, which is usually associated with a relaxed and focused mental state. For example, during meditation training, if the user can maintain this alpha wave state, it means that their concentration is good and their mind is calm. If the alpha wave power drops rapidly by more than 30% in a short period of time (such as 5 seconds), it may mean that the user is disturbed by the outside world or his or her own attention begins to disperse, and gradually leaves the focused state.

[0120] The EEG frequency characteristic parameter is the ratio of the β wave power spectrum density to the EEG signal power spectrum density. The ratio of the α wave power spectrum density (corresponding value output by the EEG module) to the total EEG signal power spectrum density (P β / Ptotal).

[0121] When the ratio is greater than 0.3 and continues to rise, it may indicate that the user is in a state of tension, excitement, or high alertness, which may be beneficial in some attention tasks that require quick responses, but if it remains at too high a level for a long time, it may lead to fatigue and difficulty in maintaining attention. For example, beta waves may rise during high-intensity cognitive tests, but if beta waves remain too high during relatively quiet tasks such as reading, it may affect the absorption and understanding of information. If the beta wave power drops sharply from a high level (such as greater than 0.4) to less than 0.2 within 10 seconds, it may suggest that the user suddenly relaxes from a highly tense state, which may be a natural reaction after an attention interruption or task completion.

[0122] The EEG frequency characteristic parameter is the ratio of the θ wave power spectrum density to the EEG signal power spectrum density. The ratio of the α wave power spectrum density (corresponding value output by the EEG module) to the total EEG signal power spectrum density (P θ / Ptotal).

[0123] When the ratio is greater than 0.15 and continues to increase, it is often associated with sleepiness, inattention or a wandering mind. For example, after a long period of study or work, if the relative intensity of theta waves gradually increases, it may mean that the user is beginning to feel tired and needs to rest and adjust, otherwise it will be difficult to maintain a high level of attention. If the theta wave power rises rapidly from a low level (such as less than 0.1) to greater than 0.2 within 20 seconds, it may indicate that the user is about to enter a stage of fatigue or inattention, and the model can issue an early warning to remind the user to adjust his or her state.

[0124] The EEG frequency characteristic parameter is the ratio of the alpha wave power spectral density to the theta wave power spectral density. Alpha waves are usually associated with a relaxed but focused state. When an individual is awake, relaxed, and not performing complex cognitive tasks, the brain generates more alpha waves. When alpha waves are high, individuals tend to feel calm and relaxed, while being able to maintain moderate attention.

[0125] Theta waves are commonly seen during periods of mild relaxation or inattention, and can also occur during sleepiness, meditation, or early sleep. Higher theta waves may indicate that an individual is distracted, sleepy, or has a wandering mind, which is not conducive to tasks that require high concentration.

[0126] The ratio of the α wave power spectral density to the θ wave power spectral density (Pα / P θ ). This ratio reflects the intensity ratio of alpha waves to theta waves and can be used to measure an individual's ability to stay focused while in a relaxed state. θ The ratio means that alpha waves are dominant, indicating that the individual is in a state of relaxation and concentration; a lower Pα / P θ A lower ratio may mean that theta waves are stronger, suggesting that the individual may feel sleepy or distracted.

[0127] As another embodiment of this implementation, the eye movement parameter is calculated based on the eye gaze point stabilization time, specifically,

[0128] The eye movement parameter is the ratio of the eye gaze point stability time to the total observation time.

[0129] Wherein, when the average fixation time on a specific visual target or area is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or,

[0130] When the saccade speed between different fixations is between 300-600° / s, and / or,

[0131] When the blinking frequency is between 10-20 times / minute,

[0132] The eye gaze point is judged to be stable.

[0133] Eye movement parameters refer to the ratio of the stable time of the eye gaze point to the total observation time. This ratio reflects the user's ability to maintain a stable gaze within a given period of time and is a key indicator for measuring visual concentration.

[0134] In order to accurately determine whether the eye gaze point is stable, the following three main conditions are set:

[0135] When the average fixation time of a user on a specific visual target or area is greater than 300 milliseconds. This means that the user is able to focus on a fixed target or area for a longer period of time without frequently shifting their gaze. Longer fixation times generally indicate higher concentration. At the same time, the average fixation time deviation between multiple targets should be less than 50 milliseconds. This ensures that the user's fixation time between different targets is relatively consistent, avoiding misjudgments due to occasional long fixation times.

[0136] When the scanning speed between different fixation points is between 300-600° / s. Saccades refer to the process of eyes moving quickly from one fixation point to another. A reasonable scanning speed range (300-600° / s) ensures that users can quickly locate new targets, but not too fast to effectively process information. This medium-speed scanning helps maintain good visual search efficiency and concentration.

[0137] When the blinking frequency is between 10-20 times / minute. A normal blinking frequency reflects the user's relaxation level and concentration state. Too high or too low blinking frequency may indicate anxiety or extreme concentration, respectively, which may affect the quality of visual input. Therefore, controlling the blinking frequency within a reasonable range helps to more accurately assess the user's concentration state.

[0138] It should be noted that the above three conditions must be met at the same time to determine that the eye gaze point is stable.

[0139] In addition, when the average fixation time on a specific visual target or area is less than 100 milliseconds or the deviation between multiple targets is greater than 200 milliseconds, it may mean that the user is just glancing quickly without deep understanding or distracted, and may be looking for other things of interest or being distracted by the surrounding environment.

[0140] When the scanning speed between different gaze points is greater than 1000° / s and lasts for multiple scanning cycles (such as more than 5 times); or when the scanning speed between different gaze points is less than 100° / s and lasts for more than 10 seconds. The former may indicate that the user is in a state of panic, anxiety or excessive distraction, unable to focus and process visual information normally; the latter may indicate that there is a problem with the user's visual system or attention, such as daze or drowsiness leading to slow eye movement.

[0141] When the blinking frequency is greater than 30 times / minute and lasts for more than 2 minutes, or when the blinking frequency is less than 5 times / minute and lasts for more than 3 minutes. The former may indicate that the user's eyes are tired, dry, or have difficulty concentrating due to external stimuli; the latter may be related to the user's over-focus, tension, or certain eye diseases, and requires further attention and analysis, because this abnormal blinking frequency may affect the acquisition of visual information and the continuation of attention.

[0142] When any of the above three conditions occurs, it is determined that the eye gaze point is unstable.

[0143] This embodiment can effectively evaluate the user's visual concentration level by setting strict stable gaze conditions and calculating the ratio of stable gaze time to total observation time. This method not only improves the accuracy of the evaluation, but also provides a scientific basis for personalizing and dynamically adjusting concentration training strategies, thereby significantly improving training effects and personal experience.

[0144] As another embodiment of this implementation, the head motion modal parameter is calculated according to the head static time, specifically:

[0145] The head motion modal parameter is the ratio of the head static time to the total observation time.

[0146] Among them, when the average acceleration of the head movement is less than 0.2m / s 2 When the average angular velocity amplitude is less than 5° / s, the head is judged to be stationary.

[0147] The head motion modality parameter refers to the ratio of the head static time to the total observation time. This ratio reflects the user's ability to keep the head relatively static within a given period of time and is an important indicator for measuring body posture control ability and attention concentration.

[0148] In order to accurately determine whether the head is still, the following two main conditions are set:

[0149] When the average acceleration of the head movement is less than 0.2 m / s 2 Acceleration is a physical quantity that describes the rate of change of an object's velocity. A small average acceleration amplitude means that the head movement is very slight and almost negligible. This shows that the user can better control his head posture and avoid unnecessary movement interference.

[0150] When the average angular velocity amplitude of the head movement is less than 5° / s. Angular velocity refers to the speed at which an object rotates around a certain axis. A lower average angular velocity amplitude indicates that the head turns very slowly and almost stays in the same direction. This further confirms that the user keeps the head stable during observation or thinking, which helps to concentrate.

[0151] When the above two conditions are met at the same time, the head is considered to be still. The head is considered to be in a state of small and stable movement. For example, when attending an important meeting in a quiet conference room, people usually keep their heads relatively stable with only slight natural adjustments. This state is conducive to concentrating on listening to others' speeches and understanding the content of the meeting.

[0152] When the average acceleration is greater than 1m / s 2 Or the average angular velocity amplitude is greater than 30° / s 2For example, in a noisy environment, people may frequently turn their heads due to surrounding disturbances, or their heads may move more significantly when they are emotionally excited (such as when arguing). In this case, it is often difficult to focus on a specific task and the attention is easily distracted by external factors.

[0153] This embodiment can effectively evaluate the user's concentration level by setting strict head stillness determination conditions and calculating the ratio of stillness time to total observation time.

[0154] As a preferred embodiment of the present invention, the concentration training method based on multimodal fusion also includes:

[0155] The concentration assessment results are displayed in real time on the concentration training interface to remind the concentration status during the concentration training process in real time;

[0156] When concentration decreases, voice or image prompts are output through the wearable device.

[0157] In this embodiment, the current concentration score is displayed to the user in an intuitive way through the display screen on the smart wearable device. Progress bars, color changes (such as green for concentration, yellow for mild distraction, and red for severe distraction), digital scores, etc. can be used. A timeline view can also be provided to show the trend of concentration changes over a period of time to help users understand their progress. It should be noted that it is necessary to ensure that the concentration score can be updated every second or every few seconds so that users can keep track of their attention status at any time.

[0158] When the system detects a decrease in concentration, a brief text prompt or suggestion will be displayed on the interface, such as "Please try to take a deep breath and relax" or "Refocus".

[0159] Specifically, the built-in speaker in the smart wearable device or the connected bone conduction headphones are used to play gentle voice reminders, such as "Please stay focused" or a guiding sentence, to help users quickly regain their focus.

[0160] Displaying specific images or animations on the display screen, such as flashing icons, gradual color changes, etc., attracts the user's attention and reminds them to adjust their status. For smart glasses equipped with AR functions, virtual prompt elements such as floating text, arrow guidance, etc. can be superimposed in the user's field of vision to guide the user on how to improve the current status.

[0161] This implementation not only improves the user's awareness of their own concentration state, but also provides timely and effective intervention to help the user maintain a good state of concentration by displaying the concentration assessment results in real time on the training interface and outputting voice or image prompts through the wearable device when the concentration decreases. This instant feedback mechanism enhances the effectiveness and interactivity of the training, providing users with more personalized support, thereby significantly improving their concentration level and overall performance.

[0162] In addition, after a single training session, the corresponding charts of continuous training results are generated based on the statistical analysis of the data of the entire training process, and detailed training result feedback is provided to users. This includes performance in each training task (such as average gaze accuracy and reaction time distribution in visual tracking games), concentration state change curve (showing the ups and downs of concentration during training), comparative analysis with past training data (such as showing the concentration score of this training compared with the average of the training scores in the past week), and personalized training suggestions for this training result (such as suggesting that users perform appropriate relaxation activities before the next training session or perform special exercises for weak links). Continuous training results generate corresponding charts to intuitively feel the training progress and provide training suggestions for a certain stage.

[0163] The present invention does not limit the storage location of the data. The detailed data of each training of the user is stored in the local database of the mobile phone, including training time, training mode, data records of each modality, concentration score, game score and other information. The local database adopts an encrypted storage method to ensure the security and privacy of user data. At the same time, local data query and backup functions are provided to facilitate users to view historical training records and perform data backup operations offline.

[0164] like Figure 3 As shown in the figure, when the mobile phone is connected to the network, the local training data is automatically synchronized to the cloud server. In the cloud, big data analysis technology is used to deeply mine and analyze massive user training data, study the distribution characteristics and change patterns of concentration of user groups of different ages, genders, occupations, etc., and provide a scientific basis for the formulation of personalized training plans. In addition, the cloud also provides data sharing and social interaction functions. Users can choose to share some of their training results (such as the best concentration score, training achievements, etc.) on social platforms, communicate and interact with other users, and increase the fun and competitiveness of training.

[0165] like Figure 2 and Figure 3As shown, a concentration training system includes a wearable device, wherein the wearable device is a smart glasses 1, and the smart glasses 1 include a lens 11, a frame 12 for fixing the lens 11, a nose pad 13 and a temple 14, the nose pad 13 is provided with an electrode 131 for collecting an electroencephalogram signal, the frame 12 is provided with a camera 121 for collecting an eye movement signal, the frame 12 or the temple 14 is provided with an inertial measurement unit 15 for collecting a head movement signal, and the frame 12 or the temple 14 is provided with a processing unit 16 for processing the electroencephalogram signal, the eye movement signal and the head movement signal, so as to adjust the concentration training strategy according to the processing result.

[0166] Specifically, the frame 12 is made of a high-strength, lightweight titanium alloy, and the nose pad 13 is made of a medical-grade silicone.

[0167] Electrode 131 uses a flexible, highly conductive dry electrode array to accurately capture EEG signals from different areas of the brain. At the same time, the wearable device is provided with a signal amplification and filtering circuit and an analog-to-digital converter (ADC) for use with electrode 131. The signal amplification and filtering circuit has an adaptive amplification adjustment function of 8000-12000 times, which can automatically adjust the amplification factor according to the strength of the EEG signal to ensure that weak signals can also be effectively amplified. The analog-to-digital converter (ADC) uses an 8-24-bit high-precision ADC, and the sampling rate can be flexibly adjusted between 100Hz-2500Hz to meet the requirements of EEG signal sampling accuracy and speed in different scenarios, ensure accurate digital conversion of EEG signals, and provide a high-quality data foundation for subsequent signal processing.

[0168] The material of the flexible electrode is more compatible with the nose pad 13, which facilitates the fixation of the electrode 131. Moreover, the nose pad 13 is located on both sides of the bridge of the nose and is a relatively stable contact point. Compared with other facial areas, the skin here is relatively flat and not easily affected by muscle movement, so good contact between the flexible electrode and the skin can be ensured. Stable contact reduces artifacts caused by slight movements of the head or changes in facial expressions, and improves the quality of the EEG signal. The electrode 131 is made of flexible materials, such as conductive polymers or graphene-based materials. These materials are not only soft and light, but also have good conductive properties and will not bring additional burden to the user. The flexible electrode can be naturally bent according to the shape of the nose pad 13, fit closely to the user's nose bridge, and provide a comfortable wearing experience.

[0169] Therefore, setting the flexible electrode on the nose pad 13 can not only improve the quality of EEG signal acquisition, but also enhance wearing comfort and user experience.

[0170] The frame 12 is provided with a camera 121 for collecting eye movement signals. The camera 121 adopts an ultra-small, high-resolution micro (MEMS) camera. The wearable device is also provided with an image processing unit used in conjunction with the camera 121. The image processing unit is based on a dedicated image signal processor (ISP) and a high-performance digital signal processor (DSP) collaborative working architecture.

[0171] By embedding the camera 121 in the frame 12, it can be directly aimed at the user's pupil and corneal reflection point to achieve accurate eye tracking. This layout helps capture high-quality eye movement data, including gaze point, scanning speed, and blinking frequency.

[0172] The frame 12 or the temple 14 is provided with an inertial measurement unit 15 for collecting head motion signals. The inertial measurement unit 15 (IMU) uses a high-precision, low-power MEMS-IMU chip, which integrates a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer.

[0173] The frame 12 or the temple 14 is provided with a processing unit 16 for processing the EEG signal, the eye movement signal, and the head movement signal. The processing unit 16 adopts a microprocessor and runs a real-time embedded Linux operating system. It can efficiently process the EEG signal, the eye movement signal, and the head movement signal data, and quickly generate corresponding control instructions and data feedback according to the preset concentration evaluation model and training logic algorithm, so as to realize the coordinated operation between the functional modules of the smart glasses 1 and the stable data interaction with the application program (APP).

[0174] Furthermore, the wearable device is also provided with a Bluetooth communication module 17, a power management module 18 and a bone conduction earphone module 19. Specifically, the Bluetooth communication module 17 complies with the Bluetooth 5.2-5.4 standard and supports Bluetooth low energy (BLE) and Bluetooth classic (BR / EDR) dual-mode communication modes. The module can automatically switch the communication mode according to the data transmission requirements to ensure that the data transmission between the smart glasses 1 and the application is efficient, stable and low-power, such as real-time transmission of EEG, eye movement, head movement data, and receiving game configuration information, training instructions, etc. sent by the application. The power management module 18 uses a rechargeable lithium polymer battery with a battery capacity of 280-1000mAh.

[0175] As a preferred embodiment of the present invention, Figure 3 As shown, the concentration training system also includes an application used in conjunction with the wearable device, and a plurality of training game modes are arranged in the application. The application can receive the concentration evaluation result obtained by the wearable device to form a training data display interface.

[0176] Specifically, the app has a variety of focus training game modes, such as the "Interstellar" mode based on visual tracking, the "Telepathy" mode controlled by EEG signals, the "Adventure Journey" mode combined with head movement interaction, and the "All-Around Challenge" mode that integrates multimodal information. Each training mode is equipped with detailed graphic instructions and difficulty level selection, and users can freely choose the appropriate mode to start training according to their interests and training goals.

[0177] The application can receive the concentration evaluation results obtained by the wearable device and form a training data display interface. The display interface presents the user's concentration data during the training process in real time in the form of visual charts, including the power spectrum of the EEG signal, the ratio change curve of the alpha wave and the beta wave, the eye movement trajectory diagram, the head movement posture animation, and the dynamic change trend of the comprehensive concentration score. At the same time, it also provides statistical analysis results of historical training data, such as weekly or monthly concentration improvement trend charts, average score comparison bar charts under various training modes, etc., so that users can intuitively understand their training progress and effects.

[0178] In addition, the application also allows users to set various parameters of the smart glasses 1, such as Bluetooth connection settings, sensor sensitivity adjustment, audio prompt volume control, etc. In addition, a user feedback portal is set up to facilitate users to feedback problems encountered during use, make improvement suggestions or share training experiences to developers, and promote continuous optimization and upgrading of products.

[0179] Anything not described in the present invention can be achieved by adopting or drawing on existing technologies.

[0180] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0181] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A concentration training method based on multimodal fusion, characterized in that: The method is applied to a concentration training system including a wearable device, comprising: During the concentration training, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one; Calculating a concentration score according to the EEG frequency characteristic parameter, the eye movement parameter, and the head movement modality parameter to evaluate concentration; Adjust your concentration training strategy based on the concentration assessment results.

2. The concentration training method based on multimodal fusion according to claim 1 is characterized in that: The concentration score is calculated according to the EEG frequency characteristic parameter, the eye movement parameter and the head movement modality parameter, specifically: Setting a first regression parameter, a second regression parameter and a third regression parameter in one-to-one correspondence with the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters, and linearly combining and superimposing the EEG frequency characteristic parameters, the eye movement parameters and the head movement modal parameters to obtain the concentration score; The first regression parameter, the second regression parameter and the third regression parameter are obtained by fitting and calculating based on the sample data marked with the concentration state.

3. The concentration training method based on multimodal fusion according to claim 1, characterized in that: Adjust the concentration training strategy based on the results of the concentration assessment, specifically: Adjust the difficulty of training tasks based on the results of the concentration assessment; and / or, providing rewards or penalties for training tasks based on the results of the concentration assessment; And / or, based on the results of the concentration assessment, complete training tasks to advance to the next level.

4. The concentration training method based on multimodal fusion according to claim 1, characterized in that: During the concentration training, the wearable device collects EEG signals, eye movement signals and head movement signals, and calculates EEG frequency characteristic parameters, eye movement parameters and head movement modal parameters one by one, specifically: The EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, the eye movement parameter is calculated based on the stabilization time of the eye gaze point, and the head movement modal parameter is calculated based on the head stillness time.

5. The concentration training method based on multimodal fusion according to claim 4 is characterized in that: The EEG frequency characteristic parameter is calculated based on the power spectrum density of a specific frequency band in the EEG signal, specifically: The electroencephalogram signal includes alpha wave signal, beta wave signal and theta wave signal. The EEG frequency characteristic parameter is the ratio of the α wave power spectrum density to the EEG signal power spectrum density, or, The EEG frequency characteristic parameter is the ratio of the β wave power spectrum density to the EEG signal power spectrum density, or, The EEG frequency characteristic parameter is the ratio of the θ wave power spectrum density to the EEG signal power spectrum density, or, The value of the EEG frequency characteristic parameter is the ratio of the α wave power spectral density to the θ wave power spectral density.

6. The concentration training method based on multimodal fusion according to claim 4 is characterized in that: The eye movement parameters are calculated based on the eye gaze point stabilization time, specifically, The eye movement parameter is the ratio of the eye gaze point stability time to the total observation time. Wherein, when the average fixation time on a specific visual target or area is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or, When the saccade speed between different fixations is between 300-600° / s, and / or, When the blinking frequency is between 10-20 times / minute, The eye gaze point is judged to be stable.

7. The concentration training method based on multimodal fusion according to claim 4 is characterized in that: The head motion modal parameters are calculated based on the head static time, specifically: The head motion modal parameter is the ratio of the head static time to the total observation time. Among them, when the average acceleration of the head movement is less than 0.2m / s 2 When the average angular velocity amplitude is less than 5° / s, the head is judged to be stationary.

8. The concentration training method based on multimodal fusion according to claim 1, characterized in that: The method further comprises, The concentration assessment results are displayed in real time on the concentration training interface to remind the concentration status during the concentration training process in real time; When concentration decreases, voice or image prompts are output through the wearable device.

9. A concentration training system, characterized in that: It includes a wearable device, which is a smart glasses. The smart glasses include lenses, a frame for fixing the lenses, a nose pad and temples. The nose pads are provided with electrodes for collecting brain electrical signals, the frame is provided with a camera for collecting eye movement signals, the frame or the temples are provided with an inertial measurement unit for collecting head movement signals, and the frame or the temples are provided with a processing unit for processing the brain electrical signals, the eye movement signals and the head movement signals, so as to adjust the concentration training strategy according to the processing results.

10. The concentration training system according to claim 9, characterized in that: It also includes an application used in conjunction with the wearable device, wherein a plurality of training game modes are arranged in the application, and the application can receive the concentration evaluation result obtained by the wearable device to form a training data display interface.

Images