A concentration training method and system based on multi-modal fusion

Abstract

The application discloses a kind of based on multi-modal fusion concentration training method and system, method includes, in the process of concentration training, brain electrical signal, eye movement signal and head movement signal are collected by wearable device, brain electrical frequency characteristic parameter, eye movement parameter and head movement mode parameter are calculated one by one correspondence;According to brain electrical frequency characteristic parameter, eye movement parameter and head movement mode parameter, concentration score is calculated to evaluate concentration;According to concentration evaluation result, adjust concentration training strategy.The application simultaneously collects brain electrical, eye movement and head movement signal by wearable device, calculates corresponding characteristic parameter and comprehensively evaluates concentration score, provides comprehensive, accurate and stable evaluation result.Based on real-time evaluation, dynamically adjust task difficulty and type, ensure challenge and interest, improve user participation and training effectiveness, promote concentration research and technological progress.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of concentration training, and specifically relates to a concentration training method and system based on multi-modal fusion. BACKGROUND

[0002] In today's digital information age, people's attention is constantly disturbed by various factors, and the decline of concentration has become a widespread problem. Traditional concentration training methods often rely on static exercises or paper-and-pencil tasks, which lack interest and have limited effectiveness, making it difficult to adapt to the fast-paced lifestyle of modern society and individual needs.

[0003] With the continuous development of sensor technology, signal processing technology and mobile application development technology, it is possible to integrate various biological detection technologies into smart wearable devices and combine them with interactive applications (APPs) to develop efficient and interesting concentration training tools.

[0004] During the concentration training process, a single modal parameter is generally used to evaluate concentration. However, the data source obtained by using only one mode is limited, and it cannot fully reflect the multi-dimensional physiological and behavioral characteristics of users. At the same time, a single mode is also easily affected by external factors, which may lead to high volatility of the evaluation results. In addition, the evaluation criteria based on a single mode are usually fixed and cannot adapt well to the unique situation of each user. For example, some people are naturally slow in eye movement response, and when only eye movement signals are used to evaluate concentration, there may be a large error.

[0005] In addition, since the concentration training task is pre-set and lacks feedback mechanisms for concentration evaluation results, the pre-set task cannot be dynamically adjusted according to the actual performance of the user during a single concentration training task. This may result in a task that is too simple and lacks challenge, or too difficult and makes the user feel frustrated. Training based on pre-set tasks often uses uniform standards, ignoring individual differences, resulting in a mismatch between training content and individual needs, greatly affecting the enthusiasm of users and the effectiveness of concentration training. SUMMARY

[0006] The present application provides a concentration training method and system based on multi-modal fusion to solve the problem of large evaluation result errors caused by using a single modal parameter for concentration evaluation during concentration training, and the problem of poor concentration training effectiveness caused by the lack of feedback mechanisms for concentration evaluation results and the inability to dynamically adjust pre-set training tasks.

[0007] The technical solution adopted by the present application is as follows:

[0008] The application discloses a concentration training method based on multi-modal fusion, which is applied to a concentration training system comprising a wearable device, and comprises the following steps:

[0009] In the concentration training process, the wearable device is used to collect electroencephalogram signals, eye movement signals and head movement signals, and electroencephalogram frequency characteristic parameters, eye movement parameters and head movement mode parameters are calculated one by one.

[0010] According to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement mode parameters, a concentration score is calculated to evaluate the concentration.

[0011] According to the concentration evaluation result, the concentration training strategy is adjusted.

[0012] According to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement mode parameters, a concentration score is calculated, specifically as follows:

[0013] The electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement mode parameters are one by one corresponding to set first regression parameters, second regression parameters and third regression parameters, and are linearly combined and superimposed to obtain the concentration score.

[0014] The first regression parameters, the second regression parameters and the third regression parameters are obtained by fitting calculation according to sample data labeled with concentration states.

[0015] According to the concentration evaluation result, the concentration training strategy is adjusted, specifically as follows:

[0016] According to the concentration evaluation result, the difficulty of the training task is adjusted.

[0017] According to the concentration evaluation result, the training task is rewarded or punished.

[0018] According to the concentration evaluation result, the training task is passed and promoted.

[0019] In the concentration training process, the wearable device is used to collect electroencephalogram signals, eye movement signals and head movement signals, and electroencephalogram frequency characteristic parameters, eye movement parameters and head movement mode parameters are calculated one by one, specifically as follows:

[0020] The electroencephalogram frequency characteristic parameters are calculated according to the power spectral density of a specific frequency band in the electroencephalogram signals, the eye movement parameters are calculated according to the eye fixation point stable time, and the head movement mode parameters are calculated according to the head stationary time.

[0021] The electroencephalogram frequency characteristic parameters are calculated according to the power spectral density of a specific frequency band in the electroencephalogram signals, specifically as follows:

[0022] The electroencephalogram signal includes alpha wave signal, beta wave signal and theta wave signal,

[0023] The electroencephalogram frequency characteristic parameter is the ratio of alpha wave power spectral density to electroencephalogram signal power spectral density, or

[0024] The electroencephalogram frequency characteristic parameter is the ratio of beta wave power spectral density to electroencephalogram signal power spectral density, or

[0025] The electroencephalogram frequency characteristic parameter is the ratio of theta wave power spectral density to electroencephalogram signal power spectral density, or

[0026] The electroencephalogram frequency characteristic parameter is the ratio of alpha wave power spectral density to theta wave power spectral density.

[0027] The eye movement parameter is calculated according to the eye gaze point stable time, specifically,

[0028] The eye movement parameter is the ratio of eye gaze point stable time to total observation time,

[0029] Wherein, when the average fixation time on a specific visual target or area is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or,

[0030] When the saccade speed between different fixation points is between 300-600° / s, and / or,

[0031] When the blink frequency is between 10-20 times per minute,

[0032] The eye gaze point is determined to be stable.

[0033] The head movement mode parameter is calculated according to the head rest time, specifically:

[0034] The head movement mode parameter is the ratio of head rest time to total observation time,

[0035] Wherein, when the average acceleration amplitude of head movement is less than 0.2m / s 2 And the average angular velocity amplitude is less than 5° / s, the head is determined to be at rest.

[0036] The method further comprises,

[0037] The concentration evaluation result is displayed in real time on the concentration training interface to remind the concentration state in the concentration training process in real time;

[0038] When the concentration decreases, the wearable device outputs voice or image prompts.

[0039] The application further provides a concentration training system, comprising a wearable device, the wearable device being smart glasses, the smart glasses comprising lenses, a frame for fixing the lenses, a nose pad and a temple, the nose pad being provided with electrodes for collecting electroencephalogram signals, the frame being provided with a camera for collecting eye movement signals, the frame or the temple being provided with an inertial measurement unit for collecting head movement signals, and the frame or the temple being provided with a processing unit for processing the electroencephalogram signals, the eye movement signals and the head movement signals to adjust a concentration training strategy according to a processing result.

[0040] The concentration training system further comprises an application program used in cooperation with the wearable device, the application program being provided with a plurality of training game modes, and the application program being capable of receiving a concentration evaluation result obtained by the wearable device to form a training data display interface.

[0041] Thanks to the above technical scheme, the application has the following beneficial effects:

[0042] 1. In the application, during the concentration training process, the electroencephalogram signals, the eye movement signals and the head movement signals are collected by the wearable device, and the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters are calculated one by one. The concentration score is calculated according to the electroencephalogram frequency characteristic parameters, the eye movement parameters and the head movement modal parameters to evaluate the concentration. By simultaneously collecting three different types of signals, i.e., electroencephalogram, eye movement and head movement, the physiological and behavioral states of the user can be more comprehensively reflected, the limitations of a single modal data source are avoided, and the accuracy of the concentration evaluation is improved. Moreover, the multiple signals complement each other, the misjudgment risk caused by the influence of a single modal on the environment or individual differences is reduced, and the evaluation result is more stable and reliable.

[0043] 2. In the application, during the training process, the data from multiple sensors can be processed and analyzed in real time, and the concentration score can be quickly given to provide timely feedback information for the concentration training. Not only can the training data of the user be tracked for a long time to establish a personal learning archive, thereby continuously optimizing the training path and providing more targeted suggestions and support, but also the specific conditions (such as physiological characteristics, psychological state, etc.) of each user can be considered, and the system can flexibly adjust the evaluation standard and the training content to better adapt to the needs of different people. At the same time, the rich multi-modal data provides valuable research materials for researchers, which helps to deeply understand the influencing factors and mechanisms of human concentration, thereby promoting the technical progress in the field of concentration research and developing more efficient concentration training methods.

[0044] 3. In the present application, the concentration training strategy is adjusted according to the concentration evaluation results. Based on the real-time evaluation results, the system can dynamically adjust the difficulty and type of the training task to ensure that each training can maintain appropriate challenge and interest, avoiding the user feeling bored or frustrated. The diversified training task design makes the whole process no longer monotonous and boring, improves the user's willingness to participate and persistence, and thus improves the effectiveness of concentration training. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0046] Figure 1 A flowchart of the concentration training method based on multi-modal fusion according to an embodiment of the present application is shown.

[0047] Figure 2 A structural diagram of the smart glasses according to an embodiment of the present application is shown.

[0048] Figure 3 A running logic diagram of the concentration training system according to an embodiment of the present application is shown.

[0049] Among them, 1 is a smart glasses; 11 is a lens; 12 is a frame; 121 is a camera; 13 is a nose pad; 131 is an electrode; 14 is a temple; 15 is an inertial measurement unit; 16 is a processing unit; 17 is a Bluetooth communication module; 18 is a power management module; 19 is a bone conduction earphone module. DETAILED DESCRIPTION

[0050] In order to more clearly illustrate the overall concept of the present application, the following will be described in detail with reference to the accompanying drawings.

[0051] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0052] As Figure 1 shown, a concentration training method based on multi-modal fusion, the method is applied to a concentration training system containing a wearable device, comprising,

[0053] S100: In the process of concentration training, the electroencephalogram signal, eye movement signal and head movement signal are collected by the wearable device, and the electroencephalogram frequency characteristic parameter, eye movement parameter and head movement modal parameter are calculated one by one.

[0054] It should be noted that the wearable device in this method is a head-mounted device, which facilitates the integration of EEG signal, eye movement signal, and head movement signal acquisition functions. Specifically, the wearable device is smart glasses 1, which utilizes augmented reality (AR) and virtual reality (VR) functions to facilitate the design and simulation of training tasks.

[0055] The smart glasses 1 are equipped with electrodes 131, which are flexible, highly conductive dry electrode arrays capable of accurately capturing electroencephalogram (EEG) signals from different areas of the brain. The smart glasses 1 also feature a miniature camera, which is an ultra-small, high-resolution MEMS camera capable of acquiring eye-tracking video images during training. Furthermore, the smart glasses 1 include an inertial measurement unit (IMU) 15, employing a high-precision, low-power MEMS-IMU chip that integrates a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, capable of acquiring head movement signals during training.

[0056] Understandably, electroencephalography (EEG) captures changes in electrical signals in the cerebral cortex, signals closely related to cognitive processes. Different frequency bands of brain waves (such as delta, theta, alpha, beta, and gamma) correspond to different psychological states and cognitive functions, such as relaxation, focus, and alertness. Numerous studies have shown that specific EEG characteristics (such as the stability of alpha waves and the activity of beta waves) can directly characterize an individual's attention level. Therefore, EEG is one of the important physiological indicators for assessing focus.

[0057] The eyes are the primary organs through which humans acquire information from the outside world, and eye movement signals can reveal the degree of attention a user pays to a specific target. By recording parameters such as fixation time, saccade length, and blink frequency, a user's visual focus can be effectively measured. Compared to internal psychological states, eye movement behavior is more intuitive and easier to measure, providing direct evidence about attention allocation. Furthermore, it can help identify which external factors may interfere with a user's focus.

[0058] Head movements reflect an individual's ability to control their body posture, and a stable head position is generally associated with higher levels of concentration. Frequent or violent head shaking may be a sign of distraction or anxiety. When a person is highly focused, they tend to remain relatively still to minimize unnecessary distractions. Therefore, monitoring head movements can help further validate the level of concentration indicated by the other two signals.

[0059] This invention, by simultaneously collecting three different types of signals—EEG, eye movement, and head motion—can more comprehensively reflect the user's physiological and behavioral state, avoiding the limitations of single-modal data sources and improving the accuracy of attention assessment. Furthermore, the complementary nature of multiple signals reduces the risk of misjudgment caused by environmental or individual differences in a single modality, making the assessment results more stable and reliable.

[0060] EEG signal acquisition, eye movement signal acquisition, and head movement signal acquisition are easier to integrate into wearable devices, facilitating the industrial layout and design of wearable devices. Of course, attention can also be assessed using various features such as heart rate variability (HRV), skin conductance response (GSR), respiratory rate, facial expression analysis, and training task completion efficiency; this invention does not limit this approach.

[0061] S200: Calculate the attention score based on the EEG frequency characteristic parameters, the eye movement parameters, and the head movement modal parameters to assess attention.

[0062] The three types of feature parameters described above are integrated to form a multi-dimensional dataset. Each feature parameter is assigned a different weight, the specific weight of which can be determined based on experimental results or expert knowledge. A comprehensive evaluation model is built using machine learning or deep learning algorithms. For example, linear regression models, support vector machines (SVM), random forests, or neural networks can be used. The goal of the model is to predict the user's attention score based on the input multimodal feature parameters. It is important to note that during model setup, a large amount of labeled attention state sample data is collected for training and validating the evaluation model. Model parameters are optimized using methods such as cross-validation to ensure its generalization ability and accuracy.

[0063] The system inputs real-time collected multimodal feature parameters into a trained evaluation model, outputting a focus score between 0 and 100. 100 represents the highest level of focus, while 0 represents complete distraction. Based on the real-time calculated focus score, the system can immediately provide feedback to the user regarding their current focus level.

[0064] S300: Adjust focus training strategies based on focus assessment results.

[0065] This method provides immediate feedback to the user's current focus level, allowing for adjustments to the difficulty and other parameters of the training task. If the user demonstrates high focus, the task complexity or speed can be increased to encourage continued high levels of concentration; conversely, if focus declines, the difficulty can be appropriately reduced, providing more prompts and support.

[0066] Alternatively, design a series of tasks that progress from easy to difficult, gradually introducing more challenging content as the user's focus increases, while maintaining an appropriate level of challenge so as not to frustrate the user.

[0067] In conclusion, by providing real-time feedback and making dynamic adjustments, training strategies can be flexibly adjusted based on attention assessment results, thereby achieving more efficient and personalized attention training. This method not only helps improve users' attention levels but also significantly enhances their performance in daily life and work.

[0068] In a preferred embodiment of the present invention, the attention score is calculated based on the EEG frequency characteristic parameters, the eye movement parameters, and the head movement modal parameters, specifically as follows:

[0069] A first regression parameter, a second regression parameter, and a third regression parameter are set one-to-one for the EEG frequency characteristic parameters, the eye movement parameters, and the head movement modal parameters. The EEG frequency characteristic parameters, the eye movement parameters, and the head movement modal parameters are linearly combined and superimposed to obtain the attention score.

[0070] The first regression parameter, the second regression parameter, and the third regression parameter are obtained by fitting and calculating based on sample data labeled with attention states.

[0071] A focus assessment model built upon big data analytics and machine learning algorithms comprehensively considers multimodal information such as EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters to provide real-time quantitative assessment of a user's focus level, aiming to minimize the error between predicted and actual values. This model establishes a mapping relationship between EEG frequency characteristic parameters, eye movement parameters, head movement modal parameters, and focus scores, enabling it to predict a user's focus state based on new input data (i.e., real-time collected user data).

[0072] Specifically, attention scores are approximated by a linear combination of multiple factors, including EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters. This model is relatively simple, easy to understand and implement, and suitable for preliminary attention assessments.

[0073] The model expression is:

[0074] y = B0 + B1X1 + B2X2 + B3X3 + z

[0075] In the formula, y is the focus score (the value can be between 0 and 100, where 0 represents complete distraction and 100 represents high focus);

[0076] X1 represents the frequency characteristic parameters of the electroencephalogram (EEG); X2 represents the eye movement parameters; X3 represents the head movement modal parameters.

[0077] B0 is the intercept, representing the baseline focus score when all independent variables are 0;

[0078] B1 is the first regression parameter; B2 is the second regression parameter; B3 is the third regression parameter, representing the degree of influence of each independent variable on the attention score;

[0079] z is the error term, used to account for other factors not included in the model and measurement errors.

[0080] By collecting a large amount of sample data labeled with attentional states (e.g., by simultaneously collecting EEG, head movement, and eye movement data from subjects in tasks with different attention requirements and manually assessing their attentional states), the values ​​of B0, B1, B2, and B3 are estimated using methods such as least squares. For example, after fitting the training data, we might obtain B0 = 30, B1 = 20, B2 = 30, and B3 = 20.

[0081] In a preferred embodiment of the present invention, the focus training strategy is adjusted based on the focus assessment results, specifically as follows:

[0082] Adjust the difficulty of training tasks based on the focus assessment results;

[0083] And / or, based on the results of attention assessment, reward or punish the training task;

[0084] And / or, based on the results of the focus assessment, conduct training tasks to advance to the next level.

[0085] In this implementation, the user's attention score is continuously monitored. EEG, eye movement, and head movement signals are collected in real time via a smart wearable device, and the current attention level is calculated. Based on changes in the attention score, the system automatically adjusts the difficulty of the training task. Attention assessment results can be categorized as follows:

[0086] High focus (score 80 or above): Increase task complexity or speed, such as speeding up target movement in visual tracking games, increasing the number of targets, or changing trajectory complexity.

[0087] Medium focus (score 50-79): Keep the existing task difficulty unchanged, but introduce some new elements to maintain the challenge.

[0088] Low focus (score below 50): Reduce task difficulty, simplify operation steps or reduce distractions, provide more prompts and support to help users refocus their attention.

[0089] This invention does not limit the focus training strategy, and any one of the following embodiments can be used, or the following embodiments can be combined in any way.

[0090] Example 1: Adjusting the difficulty of training tasks based on attention assessment results. Design various game tasks based on attention training. For example, design a visual tracking game where multiple moving targets appear on the screen, requiring players to track specific targets with their eyes and maintain focus. The game is initially set at a low difficulty level, such as slow target movement and a small number of targets. As the game progresses, the difficulty is dynamically adjusted based on the attention assessment results. If the player maintains high attention for a certain period, the number of targets is increased, the movement speed is increased, or the complexity of the movement trajectory is changed; conversely, if the player performs poorly, the difficulty is appropriately reduced to give the player more successful experiences to enhance confidence and training effectiveness.

[0091] Example 2: Rewards or penalties for training tasks based on attention assessment results. Provide users with a points system; when they successfully complete a specific attention task within a specified time, they can earn virtual points or unlock new game scenes. When users perform well, the system can provide positive feedback through voice praise, animation effects, etc., enhancing their sense of accomplishment and motivation. Simultaneously, establish weekly / monthly leaderboards to encourage user competition and strive for higher rankings; milestone achievement badges can also be set up to record users' progress.

[0092] If a user fails a task due to distraction (such as losing focus on a target or clicking incorrectly), a certain number of points are deducted or a short "penalty time" (such as pausing the game for a few seconds) is given to encourage the user to focus more. At the same time, users can be allowed to quickly recover the deducted points or shorten the penalty time through extra effort (such as completing simple auxiliary tasks) to avoid excessively damaging the user's confidence.

[0093] Example 3: Based on the focus assessment results, training tasks are used to advance through levels. The entire training process is divided into multiple levels or stages, each with different task requirements and increasing difficulty. Users need to meet certain completion standards in the current level (such as accumulating enough points or completing a specific number of tasks) to advance to the next level. Different levels are designed with different types of tasks to comprehensively train various aspects of the user's attention, such as sustained attention, selective attention, and distributed attention.

[0094] Based on the user's performance in various training tasks (such as average fixation accuracy and reaction time distribution), a comprehensive attention score is generated. Only when the score reaches a preset threshold can the user successfully advance to the next level. For levels that are not passed, the system analyzes the user's performance data and provides targeted improvement suggestions to help the user identify and improve weaknesses, thereby increasing the success rate of the next attempt.

[0095] This implementation method dynamically adjusts the difficulty of tasks, implements reward or punishment mechanisms, and allows for task progression, enabling flexible adjustments to training strategies based on focus assessment results. This results in more efficient and personalized focus training, which helps improve the user's focus level.

[0096] In a preferred embodiment of the present invention, during concentration training, the wearable device collects EEG signals, eye movement signals, and head movement signals, and calculates the EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters one-to-one, specifically as follows:

[0097] The EEG frequency characteristic parameters are calculated based on the power spectral density of a specific frequency band in the EEG signal, the eye movement parameters are calculated based on the fixation time of the eyeball, and the head movement modal parameters are calculated based on the head stillness time.

[0098] Understandably, EEG modules in smart wearable devices can capture electroencephalogram (EEG) signals generated by brain activity. These signals cover multiple frequency bands (δ, θ, α, β, γ), each corresponding to a different cognitive state.

[0099] Delta waves (0.5-4Hz) are commonly seen during deep sleep or extreme relaxation;

[0100] Theta waves (4-8 Hz) appear during periods of mild relaxation or lack of concentration;

[0101] Alpha waves (8-13Hz) are more noticeable in a relaxed but focused state;

[0102] Beta waves (13-30Hz) are active when alert, thinking, or solving problems;

[0103] Gamma waves (30-100Hz) are active at high frequencies in complex cognitive tasks.

[0104] For each frequency band, calculate its power spectral density (PSD), which is the energy distribution of the signal within that band. This can be achieved using mathematical tools such as the Fast Fourier Transform (FFT).

[0105] Calculate the ratio between specific frequency bands, such as the ratio of alpha waves to theta waves (Palpha / Ptheta), to assess the degree of relaxation and focus; or the proportion of beta waves, as an indicator of high alertness or tension.

[0106] The final output is the PSD value of each frequency band and its proportional relationship. These values ​​reflect the user's current cognitive and emotional state and are one of the important physiological indicators for assessing focus.

[0107] In addition, the camera 121 on the wearable device is used to record the position and movement trajectory of the eyes, including the duration of fixation, the length of the saccade path, and the frequency of blinking.

[0108] Identify the user's gaze behavior, i.e., the period of time the eyes remain fixed on a fixed target. Calculate the duration of each gaze and calculate the average gaze time over a period of time. Long, stable gazes generally indicate higher concentration, while frequent and short gazes may indicate inattention.

[0109] Next, inertial measurement units 15 (IMUs) on wearable devices, such as accelerometers, gyroscopes, and magnetometers, are used to monitor changes in head posture, including acceleration and angular velocity amplitude.

[0110] The system monitors head movements in real time and identifies periods of relative head stillness. It calculates the proportion of time the head remains stationary within the total observation period. Longer periods of stillness are generally associated with higher levels of focus, as a stable head posture helps reduce external distractions, allowing the user to concentrate more on the current task.

[0111] By calculating the above three characteristic parameters, the system can comprehensively capture the user's physiological and behavioral characteristics, thereby achieving a precise quantitative assessment of the user's concentration state.

[0112] As one embodiment of this implementation, the EEG frequency characteristic parameters are calculated based on the power spectral density of a specific frequency band in the EEG signal, specifically as follows:

[0113] The electroencephalogram (EEG) signals include alpha wave signals, beta wave signals, and theta wave signals.

[0114] The EEG frequency characteristic parameter is taken as the ratio of the alpha wave power spectral density to the EEG signal power spectral density, or...

[0115] The EEG frequency characteristic parameter is taken as the ratio of the β-wave power spectral density to the EEG signal power spectral density, or...

[0116] The EEG frequency characteristic parameter is taken as the ratio of the theta wave power spectral density to the EEG signal power spectral density, or...

[0117] The EEG frequency characteristic parameter is taken as the ratio of alpha wave power spectral density to the theta wave power spectral density.

[0118] The EEG frequency characteristic parameter is the ratio of the alpha wave power spectral density to the EEG signal power spectral density. The ratio of the alpha wave power spectral density (the corresponding value output by the EEG module) to the total EEG signal power spectral density (Pa / Ptotal) is calculated.

[0119] When this ratio is greater than 0.2 and fluctuates within a range of less than 0.05 over a period of time (e.g., 30 seconds), alpha waves can be considered to be in a relatively stable and strong state, usually associated with a relaxed and focused mental state. For example, if a user can maintain this alpha wave state during meditation training, it indicates good concentration and inner peace. If the alpha wave power drops rapidly by more than 30% in a short period of time (e.g., 5 seconds), it may mean that the user is experiencing external interference or that their attention is beginning to wander, gradually drifting away from a state of focus.

[0120] The EEG frequency characteristic parameter is taken as the ratio of the β-wave power spectral density to the EEG signal power spectral density. The ratio of the α-wave power spectral density (the corresponding value output by the EEG module) to the total EEG signal power spectral density (P) is calculated. β / Ptotal).

[0121] When this ratio is greater than 0.3 and continues to rise, it may indicate that the user is in a state of tension, excitement, or high alertness. This state can be beneficial in some attention tasks that require quick reactions, but if it remains too high for an extended period, it may lead to fatigue and difficulty in maintaining attention. For example, beta waves may increase during high-intensity cognitive tests, but if beta waves remain too high during relatively quiet tasks such as reading, it may affect the absorption and comprehension of information. If the beta wave power drops sharply from a high level (e.g., greater than 0.4) to less than 0.2 within 10 seconds, it may suggest that the user has suddenly relaxed from a state of high tension, possibly due to an interruption of attention or a natural reaction after the task is completed.

[0122] The EEG frequency characteristic parameter is taken as the ratio of the theta wave power spectral density to the EEG signal power spectral density. The ratio (Pα) of the alpha wave power spectral density (the corresponding value output by the EEG module) to the total EEG signal power spectral density is calculated. θ / Ptotal).

[0123] When this ratio is greater than 0.15 and continues to increase, it is often associated with drowsiness, lack of concentration, or a state of mental wandering. For example, after a long period of study or work, if the relative intensity of theta waves gradually increases, it may indicate that the user is beginning to feel fatigued and needs rest and adjustment; otherwise, it will be difficult to maintain a high level of attention. If the theta wave power rises rapidly from a low level (e.g., less than 0.1) to greater than 0.2 within 20 seconds, it may indicate that the user is about to enter a stage of fatigue or inattention, and the model can issue an early warning based on this to remind the user to adjust their state.

[0124] The EEG frequency characteristic parameter is the ratio of alpha wave power spectral density to theta wave power spectral density. Alpha waves are typically associated with a relaxed but focused state. The brain generates more alpha waves when an individual is awake, relaxed, and not performing complex cognitive tasks. Higher alpha wave levels often indicate a feeling of calm and relaxation, while maintaining adequate attention.

[0125] Theta waves are commonly seen during periods of mild relaxation or lack of concentration, and also occur during drowsiness, meditation, or early sleep stages. Higher theta waves may indicate that an individual is in a state of distraction, drowsiness, or mental wandering, which is detrimental to tasks requiring high concentration.

[0126] Calculate the ratio of α-wave power spectral density to θ-wave power spectral density (Pα / P). θ This ratio reflects the intensity of alpha waves relative to theta waves and can be used to measure an individual's ability to maintain focus in a relaxed state. A higher Pα / P ratio indicates better concentration. θ A lower Pα / P ratio indicates that alpha waves are dominant, suggesting that the individual is in a relaxed and focused state; a lower Pα / P ratio indicates that alpha waves are dominant. θ A higher ratio may indicate a stronger theta wave, suggesting that the individual may feel drowsy or have difficulty concentrating.

[0127] In another embodiment of this implementation, the eye movement parameters are calculated based on the fixation time of the eyeball, specifically as follows:

[0128] The eye movement parameters are taken as the ratio of the fixation time to the total observation time.

[0129] Specifically, when the average fixation time on a specific visual target or region is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or,

[0130] When the saccade speed between different fixation points is between 300-600° / s, and / or,

[0131] When the blinking frequency is between 10 and 20 times per minute

[0132] The fixation point was determined to be stable.

[0133] Eye movement parameters refer to the ratio of the time the eye stays still to the total observation time. This ratio reflects the user's ability to maintain a stable gaze over a given period of time and is a key indicator for measuring visual focus.

[0134] To accurately determine whether the eye's fixation point is stable, the following three main conditions were set:

[0135] A user's average fixation time on a specific visual target or area is greater than 300 milliseconds. This means the user can focus on a fixed target or area for a longer period without frequently shifting their gaze. Longer fixation times generally indicate higher concentration. Simultaneously, the average fixation time variation across multiple targets should be less than 50 milliseconds. This ensures relatively consistent fixation time across different targets, avoiding misjudgments due to occasional long fixations.

[0136] A saccade speed between different fixation points should be between 300-600° / s. Saccades refer to the rapid movement of the eyes from one fixation point to another. A reasonable saccade speed range (300-600° / s) ensures that the user can quickly locate new targets without being too fast to effectively process information. This moderate saccade speed helps maintain good visual search efficiency and concentration.

[0137] A blink rate between 10 and 20 times per minute is considered normal. A normal blink rate reflects a user's level of relaxation and concentration. Excessively high or low blink rates may indicate anxiety or extreme focus, both of which can negatively impact the quality of visual input. Therefore, maintaining a reasonable blink rate helps in more accurately assessing a user's level of focus.

[0138] It should be noted that all three conditions mentioned above must be met simultaneously for the eye's fixation point to be considered stable.

[0139] Additionally, when the average fixation time on a specific visual target or area is less than 100 milliseconds, or the deviation between multiple targets is greater than 200 milliseconds, it may indicate that the user is merely scanning quickly without in-depth understanding, or that their attention is wandering, perhaps searching for something else of interest, or being distracted by their surroundings.

[0140] When the saccade rate between different fixation points exceeds 1000° / s and persists for multiple saccade cycles (e.g., more than 5 times); or when the saccade rate between different fixation points is less than 100° / s and lasts for more than 10 seconds, the former may indicate that the user is in a state of panic, anxiety, or excessive distraction, making it impossible to focus and process visual information normally; the latter may suggest that the user has a problem with their visual system or attention, such as daydreaming or drowsiness leading to slow eye movements.

[0141] A blinking frequency greater than 30 times per minute and a duration greater than 2 minutes, or a blinking frequency less than 5 times per minute and a duration greater than 3 minutes, indicates either eye fatigue, dryness, or difficulty concentrating due to external stimuli. The former may suggest excessive focus, tension, or certain eye conditions, requiring further investigation and analysis, as this abnormal blinking frequency can affect visual information acquisition and sustained attention.

[0142] When any one of the above three conditions occurs, the eye fixation point is determined to be unstable.

[0143] This embodiment effectively assesses a user's visual attention level by setting strict stable fixation conditions and calculating the ratio of stable fixation time to total observation time. This method not only improves the accuracy of the assessment but also provides a scientific basis for personalized and dynamically adjusted attention training strategies, thereby significantly enhancing training effectiveness and individual experience.

[0144] In another embodiment of this implementation, the head motion modal parameters are calculated based on the head's resting time, specifically as follows:

[0145] The head motion modal parameters are taken as the ratio of the head stillness time to the total observation time.

[0146] Among them, when the average acceleration amplitude of head movement is less than 0.2 m / s² 2 Furthermore, if the average angular velocity amplitude is less than 5° / s, the head is considered stationary.

[0147] Head movement modal parameters refer to the ratio of head stillness time to total observation time. This ratio reflects the user's ability to keep their head relatively still within a given time period and is an important indicator for measuring body posture control and attention concentration.

[0148] To accurately determine whether the head is still, the following two main conditions are set:

[0149] When the average acceleration amplitude of head movement is less than 0.2 m / s² 2 Acceleration is a physical quantity that describes the rate of change of an object's velocity. A small average acceleration amplitude means that head movements are very slight, almost negligible. This indicates that the user can control their head posture well and avoid unnecessary movement interference.

[0150] When the average angular velocity of head movement is less than 5° / s, it indicates that the head is rotating very slowly, almost in the same direction. Angular velocity refers to the speed at which an object rotates around an axis. This further confirms that the user has maintained head stability during observation or thinking, which helps with concentration.

[0151] When both of the above conditions are met simultaneously, the head is considered stationary. This is considered a state of minimal and stable head movement. For example, when attending an important meeting in a quiet conference room, people usually keep their heads relatively stable with only slight natural adjustments. This state is conducive to concentrating on listening to others and understanding the meeting content.

[0152] When the average acceleration amplitude is greater than 1 m / s² 2 Or the average angular velocity amplitude is greater than 30° / s 2This is considered excessive head movement. For example, in a noisy environment, people may frequently turn their heads due to surrounding distractions, or their heads may move significantly when they are emotionally agitated (such as during an argument). In such cases, it is often difficult to concentrate on a specific task, and attention is easily distracted by external factors.

[0153] This embodiment can effectively assess a user's level of concentration by setting strict criteria for determining head stillness and calculating the ratio of stillness time to total observation time.

[0154] As a preferred embodiment of the present invention, the attention training method based on multimodal fusion further includes,

[0155] The focus assessment results are displayed in real time on the focus training interface to provide real-time reminders of focus status during the focus training process;

[0156] When concentration decreases, the wearable device outputs voice or image prompts.

[0157] In this embodiment, the current focus score is displayed to the user intuitively via a screen on the smart wearable device. This can be achieved using progress bars, color changes (e.g., green for focused, yellow for slightly distracted, and red for severely distracted), or numerical scores. A timeline view can also be provided to show the trend of focus changes over a period of time, helping users understand their progress. It is important to note that the focus score needs to be updated every second or every few seconds so that users can monitor their attention levels at any time.

[0158] When the system detects a decline in concentration, it displays brief text prompts or suggestions on the interface, such as "Try taking deep breaths to relax" or "Refocus your attention."

[0159] Specifically, by using the built-in speakers in smart wearable devices or connected bone conduction headphones, gentle voice reminders such as "Please pay attention and stay focused" or a guided statement can be played to help users quickly regain focus.

[0160] Displaying specific images or animations on the screen, such as flashing icons or gradient color changes, can attract the user's attention and remind them to adjust their state. For smart glasses equipped with AR functionality, virtual prompts, such as floating text or arrows, can be overlaid in the user's field of vision to guide them on how to improve their current state.

[0161] This implementation displays the attention assessment results on the training interface in real time and provides voice or image prompts via wearable devices when attention wanes. This not only improves users' awareness of their own attention status but also allows for timely and effective intervention to help them maintain a good level of focus. This instant feedback mechanism enhances the effectiveness and interactivity of training, providing users with more personalized support, thereby significantly improving their attention levels and overall performance.

[0162] In addition, after each training session, statistical analysis of the data throughout the training process generates corresponding charts of continuous training results, providing users with detailed feedback. This includes performance in various training tasks (such as average fixation accuracy and reaction time distribution in visual tracking games), attention state change curves (showing fluctuations in attention during training), comparative analysis with past training data (such as comparing the attention score of this training session with the average score of training over the past week), and personalized training suggestions based on the training results (such as suggesting appropriate relaxation activities or targeted practice for weak areas before the next training session). The continuous training results generate corresponding charts to intuitively illustrate the training progress and provide training suggestions for certain stages.

[0163] This invention does not limit the storage location of the data. Detailed data from each user training session is stored in a local database on the phone, including training time, training mode, data records for each modality, attention score, game score, and other information. The local database uses encrypted storage to ensure the security and privacy of user data. Simultaneously, local data query and backup functions are provided, allowing users to view historical training records and perform data backup operations offline.

[0164] like Figure 3 As shown, when the phone connects to the network, it automatically synchronizes local training data to the cloud server. In the cloud, big data analytics are used to deeply mine and analyze massive amounts of user training data, studying the distribution characteristics and changing patterns of attention spans among user groups of different ages, genders, and occupations, providing a scientific basis for developing personalized training programs. Furthermore, the cloud provides data sharing and social interaction functions. Users can choose to share some of their training results (such as best attention scores, training achievements, etc.) on social platforms to communicate and interact with other users, increasing the fun and competitiveness of training.

[0165] like Figure 2 and Figure 3As shown, a focus training system includes a wearable device, namely smart glasses 1. The smart glasses 1 includes lenses 11, a frame 12 for fixing the lenses 11, a nose pad 13, and temples 14. The nose pad 13 is provided with electrodes 131 for collecting electroencephalogram (EEG) signals. The frame 12 is provided with a camera 121 for collecting eye movement signals. The frame 12 or the temples 14 are provided with an inertial measurement unit 15 for collecting head movement signals. The frame 12 or the temples 14 are provided with a processing unit 16 for processing the EEG signals, the eye movement signals, and the head movement signals, so as to adjust the focus training strategy according to the processing results.

[0166] Specifically, the frame 12 is made of high-strength, lightweight titanium alloy, and the nose pad 13 is made of medical-grade silicone.

[0167] Electrode 131 employs a flexible, highly conductive dry electrode array to precisely capture EEG signals from different brain regions. The wearable device also includes signal amplification and filtering circuitry and an analog-to-digital converter (ADC) that work in conjunction with electrode 131. The signal amplification and filtering circuitry features an adaptive amplification factor adjustment function of 8000-12000 times, automatically adjusting the amplification factor according to the strength of the EEG signal to ensure effective amplification even for weak signals. The ADC uses an 8-24 bit high-precision ADC with a sampling rate that can be flexibly adjusted between 100Hz and 2500Hz to meet the requirements of EEG signal sampling accuracy and speed in different scenarios, ensuring accurate digital conversion of EEG signals and providing a high-quality data foundation for subsequent signal processing.

[0168] The flexible electrodes are better suited to the material of the nose pad 13, facilitating the fixation of the electrodes 131. Furthermore, the nose pad 13, located on both sides of the bridge of the nose, provides a relatively stable contact point. Compared to other facial areas, the skin here is flatter and less affected by muscle movement, thus ensuring good contact between the flexible electrodes and the skin. Stable contact reduces artifacts caused by slight head movements or facial expressions, improving the quality of the EEG signal. The electrodes 131 are made of flexible materials, such as conductive polymers or graphene-based materials, which are not only soft and lightweight but also possess good conductivity, without placing any extra burden on the user. The flexible electrodes can naturally bend according to the shape of the nose pad 13, closely conforming to the user's nose bridge for a comfortable wearing experience.

[0169] Therefore, placing flexible electrodes on the nose pad 13 can not only improve the quality of EEG signal acquisition, but also enhance wearing comfort and user experience.

[0170] The frame 12 is equipped with a camera 121 for acquiring eye movement signals. The camera 121 is an ultra-small, high-resolution micro (MEMS) camera. The wearable device also includes an image processing unit that works in conjunction with the camera 121. The image processing unit is based on a collaborative architecture of a dedicated image signal processor (ISP) and a high-performance digital signal processor (DSP).

[0171] By embedding the camera 121 into the frame 12, it can be directly aimed at the user's pupil and corneal reflection point, enabling precise gaze tracking. This layout helps capture high-quality eye movement data, including fixation point, saccade speed, and blink frequency.

[0172] The frame 12 or the temple 14 is provided with an inertial measurement unit 15 for collecting head motion signals. The inertial measurement unit 15 (IMU) uses a high-precision, low-power MEMS-IMU chip and integrates a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer.

[0173] The frame 12 or the temple 14 is provided with a processing unit 16 for processing the electroencephalogram (EEG) signals, eye movement signals, and head movement signals. The processing unit 16 employs a microprocessor and runs on a real-time embedded Linux operating system. It can efficiently process EEG, eye movement, and head movement signal data, and quickly generate corresponding control commands and data feedback based on a preset attention assessment model and training logic algorithm, thereby achieving coordinated operation between the various functional modules of the smart glasses 1 and stable data interaction with the application program (APP).

[0174] Furthermore, the wearable device also includes a Bluetooth communication module 17, a power management module 18, and a bone conduction headphone module 19. Specifically, the Bluetooth communication module 17 conforms to the Bluetooth 5.2-5.4 standard and supports dual-mode communication of Bluetooth Low Energy (BLE) and Bluetooth Classic (BR / EDR). This module can automatically switch communication modes according to data transmission requirements, ensuring efficient, stable, and low-power data transmission between the smart glasses 1 and the application, such as real-time transmission of EEG, eye-tracking, and head-movement data, as well as receiving game configuration information and training instructions sent by the application. The power management module 18 uses a rechargeable lithium polymer battery with a capacity of 280-1000mAh.

[0175] As a preferred embodiment of the present invention, such as Figure 3 As shown, the focus training system also includes an application used in conjunction with the wearable device. The application has multiple training game modes and can receive focus assessment results obtained from the wearable device to form a training data display interface.

[0176] Specifically, the application features a variety of focus training game modes, such as the "Interstellar Travel" mode based on visual tracking, the "Telepathy" mode controlled by EEG signals, the "Adventure" mode combining head movement interaction, and the "All-Round Challenge" mode integrating multimodal information. Each training mode comes with detailed text and image instructions and difficulty level options, allowing users to freely choose the appropriate mode to begin training based on their interests and training goals.

[0177] The application can receive the attention assessment results from the wearable device and generate a training data display interface. This interface presents the user's attention data during training in real-time using visual charts, including power spectrum graphs of EEG signals, curves showing the ratio of alpha waves to beta waves, eye movement maps, head movement animations, and dynamic trends in the overall attention score. It also provides statistical analysis results of historical training data, such as weekly or monthly attention improvement trend charts and bar charts comparing average scores across different training modes, allowing users to intuitively understand their training progress and effectiveness.

[0178] In addition, the application allows users to configure various parameters of the Smart Glasses 1, such as Bluetooth connection settings, sensor sensitivity adjustment, and audio prompt volume control. Furthermore, a user feedback portal is provided, allowing users to report problems encountered during use, offer suggestions for improvement, or share training experiences with the developers, thus promoting continuous product optimization and upgrades.

[0179] For any parts not mentioned in this invention, existing technologies can be used or referenced.

[0180] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0181] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A focus training method based on multimodal fusion, characterized in that, The method is applied to a focus training system incorporating a wearable device, wherein the wearable device is smart glasses, and includes... During focus training, the wearable device collects electroencephalogram (EEG) signals, eye movement signals, and head movement signals, and calculates the corresponding EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters. The EEG frequency characteristic parameters are calculated based on the power spectral density of the alpha, beta, and theta wave signals within the EEG signals. The eye movement parameters are calculated based on the fixation time of the eyeball, and are taken as the ratio of the fixation time of the eyeball to the total observation time. The head motion modal parameters are calculated based on the head stillness time and are taken as the ratio of the head stillness time to the total observation time. Based on the EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters, a first regression parameter, a second regression parameter, and a third regression parameter are set one-to-one. The EEG frequency characteristic parameters, eye movement parameters, and head movement modal parameters are linearly combined and superimposed to calculate the attention score for evaluating attention. The first regression parameter, the second regression parameter, and the third regression parameter are obtained by least squares fitting based on sample data labeled with attention states. Based on the focus assessment results, adjust the focus training strategy and display the focus assessment results in real time on the focus training interface to provide real-time reminders of the focus status during the focus training process; The eye movement parameters are calculated based on the fixation time of the eye's gaze point, specifically, When the average fixation time on a fixed visual target or region is greater than 300 milliseconds and the deviation between multiple targets is less than 50 milliseconds, and / or, When the saccade speed between different fixation points is between 300-600° / s, and / or, When the blinking frequency is between 10 and 20 times per minute The fixation point is determined to be stable. The head motion modal parameters are calculated based on the head's rest time, specifically as follows: When the average acceleration amplitude of head movement is less than 0.2 m / s² 2 Furthermore, if the average angular velocity amplitude is less than 5˚ / s, the head is considered stationary.

2. The attention training method based on multimodal fusion according to claim 1, characterized in that, Based on the focus assessment results, the focus training strategy was adjusted as follows: Adjust the difficulty of training tasks based on the focus assessment results; And / or, based on the results of attention assessment, reward or punish the training task; And / or, based on the results of the focus assessment, conduct training tasks to advance to the next level.

3. The attention training method based on multimodal fusion according to claim 1, characterized in that, The EEG frequency characteristic parameters are calculated based on the power spectral density of the alpha, beta, and theta waves in the EEG signal, specifically as follows: The EEG frequency characteristic parameter is taken as the ratio of the alpha wave power spectral density to the EEG signal power spectral density, or... The EEG frequency characteristic parameter is taken as the ratio of the β-wave power spectral density to the EEG signal power spectral density, or... The EEG frequency characteristic parameter is taken as the ratio of the theta wave power spectral density to the EEG signal power spectral density, or... The EEG frequency characteristic parameter is taken as the ratio of alpha wave power spectral density to the theta wave power spectral density.

4. The attention training method based on multimodal fusion according to claim 1, characterized in that, The method also includes, When concentration decreases, the wearable device outputs voice or image prompts.

5. A concentration training system, characterized in that, Used to perform the attention training method based on multimodal fusion as described in any one of claims 1 to 4 The invention includes wearable devices, specifically smart glasses. The smart glasses include lenses, a frame for fixing the lenses, nose pads, and temples. The nose pads are equipped with electrodes for collecting electroencephalogram (EEG) signals. The frame is equipped with a camera for collecting eye-tracking signals. The frame or temples are equipped with an inertial measurement unit for collecting head-tracking signals. The frame or temples are equipped with a processing unit for processing the EEG signals, eye-tracking signals, and head-tracking signals to adjust attention training strategies based on the processing results.

6. The concentration training system according to claim 5, characterized in that, It also includes an application that works with the wearable device, which has multiple training game modes and can receive attention assessment results from the wearable device to form a training data display interface.

Images