Heart flow experience closed-loop regulation and control method and system based on sensory motion synchronization

Abstract

The invention discloses a cardiac flow experience closed-loop regulation and control method and system based on sensory movement synchronization. The method comprises the following steps: acquiring initial electroencephalogram data of a user, extracting frequency domain periodic and non-periodic electroencephalogram characteristics related to cardiac flow experience to obtain an initial electroencephalogram characteristic value, randomly presenting rhythmic sensory stimulation at a preset rhythm, and executing corresponding behavior response by the user; calculating a sensory motion synchronization index; according to the sensory motion synchronism index and a preset interval, the sensory motion synchronism index is stabilized in the preset interval by adjusting the rhythm; extracting frequency domain periodic and non-periodic electroencephalogram features again, and calculating the variable quantity of the relative initial value; judging whether the sensory stimulation presentation rhythm is adjusted or not in combination with the variable quantity and a preset standard; and if the rhythm needs to be adjusted, repeatedly executing the previous steps until the variable quantity of each electroencephalogram characteristic value is adjusted to a preset standard. The system can adaptively adjust the interaction content, optimizes the heart stream experience according to the electroencephalogram characteristics of the user, and improves the immersion and pleasure of the user.

Description

Technical Field

[0001] The present invention relates to the field of medical health and human-computer interaction technology, and in particular to a closed-loop control method and system for flow experience based on sensory-motor synchronization. Background Art

[0002] Flow experience is a psychological state of high concentration, immersion and pleasure; frequent flow experience can relieve stress, enhance the immune system and improve health. Previous studies have shown that sensory-motor rhythm synchronization can promote flow experience. However, the existing system lacks dynamic detection and adjustment of individual EEG characteristics and sensory-motor synchronization of users, and fails to fully utilize rhythm synchronization to optimize the flow state, thus limiting the effectiveness of flow experience. Summary of the invention

[0003] In order to solve the lack of dynamic detection and adjustment of EEG characteristics and sensorimotor synchronization in the prior art, the present invention provides a closed-loop control method and system for flow experience based on sensorimotor synchronization. The system can adaptively adjust the interactive content, optimize the flow experience according to the user's EEG characteristics, and enhance the user's sense of immersion and pleasure.

[0004] The specific scheme adopted is as follows:

[0005] On the one hand, the present invention provides a closed-loop control method for flow experience based on sensory-motor synchronization.

[0006] Step 1, obtaining the user's initial EEG data;

[0007] Step 2: extract frequency domain periodic and non-periodic EEG features related to flow experience to obtain initial EEG feature values;

[0008] Step 3, randomly presenting rhythmic sensory stimulation at a predetermined rhythm, and the user generates a corresponding rhythmic behavioral response in combination with the rhythmic sensory stimulation;

[0009] Step 4, calculating the sensorimotor synchronization index between the user behavior and the sensory stimulus presentation rhythm;

[0010] Step 5, when the sensory-motor synchronization index exceeds the preset interval, adjusting the sensory stimulation presentation rhythm so that the sensory-motor synchronization index is stabilized within the preset interval;

[0011] Step 6, extract frequency domain periodic and non-periodic EEG features related to flow experience again, calculate the change between each EEG feature value and the initial EEG feature value, otherwise, proceed to the next step;

[0012] Step 7, combining the obtained change amount and the preset standard, judging whether the presentation rhythm of the sensory stimulus should be adjusted;

[0013] Step 8: If the obtained change exceeds the preset standard, adjust the rhythm of the rhythmic sensory stimulation presentation rhythm, and repeat steps 4 to 7 until the change of each EEG characteristic value is adjusted to the preset standard.

[0014] Furthermore, the specific method of step 2 is:

[0015] Perform bandpass filtering and notch filtering preprocessing on the collected EEG data;

[0016] The preprocessed EEG data were segmented according to the preset time periods, eye movements and electromyography were eliminated using principal component analysis, and all EEG data were re-referenced to the bilateral mastoids;

[0017] Calculate and extract the spontaneous neural oscillation energy in theta, alpha, beta, and gamma frequency bands as the periodic initial EEG characteristic values;

[0018] The frequency domain features of the EEG data are decomposed into periodic and non-periodic components, and the non-periodic index value of each electrode is extracted as the initial non-periodic EEG feature value.

[0019] Furthermore, in step 3, a music player is used to provide the user with a rhythmic music melody with an initial presentation rhythm of 60-90 beats / minute, and a touch screen is used to provide visual cues and tactile feedback that are phase-synchronized with the rhythmic music melody, and the user performs rhythmic tapping responses on the touch screen according to the visual cues.

[0020] Furthermore, in step 4, the time difference method is used to calculate the time point S at which each visual cue is presented. i The tapping movement time point R i The time difference between i ; Calculate the average value of the time difference at regular intervals to evaluate the user's sensory-motor synchronization.

[0021] Preferably, the preset interval set in step 5 is [-100, 100] ms.

[0022] Preferably, in step 7, when the change in power spectrum density of the obtained parietal-occipital gamma increases and the change in the non-periodic exponent decreases, the current music presentation rhythm is maintained until the user chooses to exit; otherwise, the sensory stimulation presentation rhythm is adjusted based on the initial rhythm.

[0023] On the other hand, the present invention also provides a closed-loop control system for flow experience based on sensory-motor synchronization, the system comprising an EEG data acquisition device and a data processing terminal connected to the device, the EEG data acquisition device collects the user's initial EEG signal and the EEG signal after sensory-motor synchronization, and transmits them to the data processing terminal, the data processing terminal is provided with the following modules:

[0024] An extraction module is used to extract frequency-domain periodic and non-periodic EEG features related to flow experience in EEG signals;

[0025] A music player for presenting rhythmic music melodies;

[0026] A touch screen for presenting visual cues that are phase-synchronized with the rhythmic music melody and tactile feedback formed by the user's tapping response;

[0027] The sensorimotor synchronization analysis module is used to calculate the sensorimotor synchronization index between the user behavior and the sensory stimulation presentation rhythm; when the obtained sensorimotor synchronization index exceeds the preset interval, it is judged as poor synchronization, and the music melody of the music player is adjusted to stabilize the sensorimotor synchronization index within the preset interval;

[0028] A heart flow calculation module is used to calculate the change of each EEG characteristic value relative to the initial EEG characteristic value after the sensory-motor synchronization index is stabilized;

[0029] The feedback optimization module maintains the current music presentation rhythm of the music player when the change in each EEG characteristic value of the user reaches a preset standard; otherwise, based on the initial music melody, the presentation rhythm of the sensory stimulation is changed within a reasonable range with a certain rhythm as the step length, and then the sensorimotor synchronization analysis module is used to perform sensorimotor synchronization analysis and the change in each EEG characteristic value is calculated by the flow calculation module.

[0030] The technical solution of the present invention has the following advantages:

[0031] A. The present invention combines the flow experience regulation of sensorimotor synchronization and EEG characteristics. It uses sensorimotor rhythm synchronization to induce flow experience for the first time, and uses periodic and non-periodic signals in the user's EEG frequency domain to evaluate the flow state. Whether the change between the EEG characteristics corresponding to the flow state and the initial EEG characteristics exceeds the preset standard is used as the basis for further adjusting the sensorimotor rhythm. A real-time EEG feedback closed-loop control system based on rhythm synchronization adjustment is realized, thereby enhancing the user's concentration, creativity and pleasure.

[0032] B. The system of the present invention dynamically adjusts the rhythm and melody according to the user's sensory-motor synchronization and EEG characteristics, and optimizes the flow experience using a closed-loop adjustment mechanism based on real-time behavioral data and physiological data, rather than a traditional fixed rhythm or purely algorithm-generated rhythm scheme. It provides a non-invasive flow induction scheme based on rhythm synchronization, which can be expanded to virtual reality, games, rehabilitation training and other fields, and has significant market value and social benefits.

[0033] C. The system of the present invention integrates tactile, visual and auditory rhythmic stimulation, and performs personalized feedback optimization based on the user's EEG signals and sensory-motor synchronization indicators. When the flow does not reach the enhancement standard, the beat rate is adaptively adjusted to continuously maintain the flow experience through multiple rounds of optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific embodiments of the present invention, the drawings required for use in the specific embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is a flow chart of the closed-loop control method of flow experience based on sensory-motor synchronization provided by the present invention;

[0036] Figure 2 This is a composition diagram of the closed-loop control system for flow experience based on sensory-motor synchronization provided by the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 As shown, the present invention provides a closed-loop control method for flow experience based on sensory-motor synchronization, and the method comprises the following steps:

[0039]

S01

[0040]

S02

[0041]

S021

[0042] [S022] Segment the preprocessed EEG data according to a preset time period (e.g., 2s period), use principal component analysis to remove eye movements and electromyography, and re-reference all EEG data to the bilateral mastoids;

[0043]

S023

[0044] For example, common frequency domain analysis methods, such as the Welch method, are used to calculate and extract the energy amplitude or the average value of the power spectrum in different frequency bands of the user (such as theta: 4-8Hz; alpha: 8-13Hz; beta: 13-30Hz; gamma: 30-90Hz) as the initial EEG characteristic value of the high-frequency periodicity of spontaneous EEG neural oscillations.

[0045]

S024

[0046] For example, FOOOF (Fitting Oscillations and One-Over-F) decomposes the frequency domain features into periodic and non-periodic components, and extracts the non-periodic index value of each electrode of the user as the non-periodic initial EEG characteristic value of spontaneous EEG neural oscillation.

[0047] [S03] Rhythmic sensory stimulation is randomly presented at a predetermined rhythm, and the user generates a behavioral response of a corresponding rhythm in combination with the rhythmic sensory stimulation.

[0048] The present invention utilizes a music player and a touch screen. The music player provides a rhythmic music melody with an initial presentation rhythm of 60-90 beats / minute to the user, and provides phase-synchronized visual prompts and tactile feedback on the touch screen. The user performs rhythmic tapping responses on the touch screen according to the visual prompts.

[0049]

S04

[0050] Preferably, the time difference method is used to calculate the time point S at which each visual cue is presented. i The tapping movement time point R i The time difference between i =R i -S i , and then calculate the average value of the time difference every certain period of time (such as 10-30s) Assess user sensorimotor synchronization.

[0051] When the average value of the time difference is not within a reasonable interval, such as [-100, 100] ms, the user's sensory-motor synchronization is considered to be poor, otherwise, the user's sensory-motor synchronization is considered to be good. The preset interval used here can be [-50 50] ms for generally healthy people, and for the elderly, the interval can be appropriately expanded, such as [100, 100] ms.

[0052] [S05] When the sensory-motor synchronization index exceeds the preset interval, the sensory stimulation presentation rhythm (i.e., the number of beats in one minute) is adjusted to make it close to the user's tapping frequency (i.e., the number of tappings in one minute), that is, the sensory-motor synchronization index is stabilized in the preset interval. Of course, when the sensory-motor synchronization is good, there is no need to adjust the presentation rate of the rhythmic music melody, and the next step [S06] is directly executed.

[0053]

S06

[0054] After the sensorimotor synchronization is stable, the frequency domain periodic and non-periodic EEG features related to the flow experience are extracted again, such as Period j With Aperiod j , and calculate Period separately j 、Aperiod j With Period 初始、 Aperiod 初始 The difference between and is used to obtain the corresponding changes.

[0055] [S07] Combine the obtained change amount and its preset standard to determine whether to adjust the presentation rhythm of the sensory stimulation. When the change amount of the EEG feature is determined to be within the preset standard, for example, when the power spectrum density of the user's parietal-occipital gamma increases (such as 1dB) and the non-periodic index decreases (such as 0.1dB), the presentation rhythm of the current music in the music player can be maintained until the user chooses to exit, otherwise, execute the next step.

[0056] [S08] If the obtained change exceeds the preset standard, the presentation rhythm of the rhythmic music melody and the phase-synchronized visual prompt is adjusted, and [S04] to [S07] are repeated until the change of each EEG characteristic value is adjusted to the preset standard. For example, with a certain frequency as the step length (such as 5bmp), within a certain rhythm range, such as 60-120bmp, the presentation rhythm of the rhythmic music melody and the phase-synchronized visual prompt is changed.

[0057] The present invention realizes the synchronous interaction of music, action and user status by detecting and analyzing the user's EEG periodic and non-periodic signals and sensory-motor synchronization, can adaptively adjust the interactive content, optimize the flow experience according to the user's EEG characteristics, and enhance the user's immersion and pleasure.

[0058] like Figure 2As shown, the present invention also provides a closed-loop control system for flow experience based on sensory-motor synchronization, including an EEG data acquisition device and a data processing terminal connected to the device, wherein the EEG data acquisition device acquires the user's initial EEG signal and the EEG signal after sensory-motor synchronization, and transmits the data to the data processing terminal for processing, wherein the data processing terminal is provided with an extraction module, a music player, a touch screen, a sensory-motor synchronization analysis module, a flow calculation module, and a feedback optimization module. The extraction module is used to extract frequency domain periodic and non-periodic EEG features related to flow experience in the EEG signal, such as; the music player is used to present rhythmic music melodies, and music of different melodies can be selected; the touch screen is used to present visual prompts that are phase-synchronized with the rhythmic music melody and tactile feedback formed by the user's tapping reaction, the music melody presented by the music player can be presented in a visual prompting manner, and the user performs a touch screen tapping action according to the music heard and the visual prompt seen, and the user's tapping action reaction will cause the problem of incomplete synchronization with the generated music melody. The sensorimotor synchronization analysis module can calculate the sensorimotor synchronization index between the user's tapping behavior and the rhythm of sensory stimulation presentation, that is, calculate the time difference between each visual prompt presentation moment and the tapping movement moment, and calculate the average value of the time difference every certain period of time to evaluate the user's sensorimotor synchronization; when the obtained sensorimotor synchronization index exceeds the preset interval, it is judged as poor synchronization, and the music melody of the music player is automatically adjusted to stabilize the sensorimotor synchronization index within the preset interval. The flow calculation module is used to calculate the change in each EEG characteristic value relative to the initial EEG characteristic value after the sensorimotor synchronization index is stabilized. That is to say, when the user's sensorimotor synchronization index reaches the preset interval, the change in the EEG characteristic value at this time relative to the initial EEG characteristic value is further calculated. The feedback optimization module compares the change of each EEG characteristic value with the preset standard and judges whether to adjust the current music rhythm. When the change of each EEG characteristic value of the user reaches the preset standard, the current music presentation rhythm of the music player is maintained; otherwise, based on the starting music melody, the presentation rhythm of the sensory stimulation is changed within a reasonable range with a certain rhythm as the step size, and then the sensory-motor synchronization analysis module is used to perform sensory-motor synchronization analysis, and the change of each EEG characteristic value is calculated through the flow calculation module, and then the flow experience is optimized through the feedback optimization module.

[0059] The present invention is the first to use sensorimotor rhythm synchronization to induce flow experience, and uses periodic and non-periodic signals in the user's EEG frequency domain to evaluate the flow state, forming a real-time EEG feedback closed-loop control system based on rhythm synchronization adjustment. The system integrates rhythmic stimulation of tactile, visual, and auditory multimodal feedback, and performs personalized feedback optimization based on the user's EEG signals and sensorimotor synchronization indicators. When the flow does not meet the enhancement standard, the beat rate is adaptively adjusted, and the flow experience is continuously maintained through multiple rounds of optimization, thereby enhancing the user's concentration, creativity, and pleasure. It can be expanded to virtual reality, games, rehabilitation training and other fields.

[0060] The matters not described in the present invention are applicable to the prior art.

[0061] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. However, the obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A closed-loop control method for flow experience based on sensory-motor synchronization, characterized in that: Step 1, obtaining the user's initial EEG data; Step 2: extract frequency domain periodic and non-periodic EEG features related to flow experience to obtain initial EEG feature values; Step 3, randomly presenting rhythmic sensory stimulation at a predetermined rhythm, and the user generates a corresponding rhythmic behavioral response in combination with the rhythmic sensory stimulation; Step 4, calculating the sensorimotor synchronization index between the user behavior and the sensory stimulus presentation rhythm; Step 5, when the sensorimotor synchronization index exceeds the preset interval, adjusting the sensory stimulation presentation rhythm so that the sensorimotor synchronization index is stabilized within the preset interval; Step 6, extracting frequency domain periodic and non-periodic EEG features related to flow experience again, and calculating the change between each EEG feature value and the initial EEG feature value; Step 7, combining the obtained change amount and the preset standard, judging whether the presentation rhythm of the sensory stimulus should be adjusted; Step 8: If the obtained change exceeds the preset standard, adjust the rhythm of the rhythmic sensory stimulation presentation rhythm, and repeat steps 4 to 7 until the change of each EEG characteristic value is adjusted to the preset standard.

2. The closed-loop control method for flow experience based on sensory-motor synchronization according to claim 1 is characterized in that: The specific method of step 2 is: Perform bandpass filtering and notch filtering preprocessing on the collected EEG data; The pre-processed EEG data were segmented according to the preset time periods, eye movements and electromyography were eliminated using principal component analysis, and all EEG data were re-referenced to the bilateral mastoids; Calculate and extract the spontaneous neural oscillation energy in theta, alpha, beta, and gamma frequency bands as the periodic initial EEG characteristic values; The frequency domain features of the EEG data are decomposed into periodic and non-periodic components, and the non-periodic index value of each electrode is extracted as the initial non-periodic EEG feature value.

3. The closed-loop control method for flow experience based on sensory-motor synchronization according to claim 1, characterized in that: In step 3, a music player is used to provide the user with a rhythmic music melody with an initial presentation rhythm of 60-90 beats / minute, and a touch screen is used to provide visual cues and tactile feedback that are phase-synchronized with the rhythmic music melody. The user performs rhythmic tapping responses on the touch screen according to the visual cues.

4. The closed-loop control method for flow experience based on sensory-motor synchronization according to claim 3 is characterized in that: In step 4, the time difference method is used to calculate the time point S of each visual cue presentation. i The tapping movement time point R i The time difference between i ; Calculate the average value of the time difference at regular intervals to evaluate the user's sensory-motor synchronization.

5. The closed-loop control method for flow experience based on sensory-motor synchronization according to claim 1, characterized in that: The preset interval set in step 5 is [-100, 100] ms.

6. According to the closed-loop control method of flow experience based on sensorimotor synchronization according to claim 1, in step 7, when the power spectral density change of the obtained parietal-occipital gamma of the brain increases and the non-periodic exponential change decreases, the current music presentation rhythm is maintained until the user chooses to exit; otherwise, the sensory stimulation presentation rhythm is adjusted based on the initial rhythm.

7. A closed-loop control system for flow experience based on sensorimotor synchronization, the system comprising an EEG data acquisition device and a data processing terminal connected to the device, the EEG data acquisition device collects the user's initial EEG signal and the EEG signal after sensorimotor synchronization, and transmits them to the data processing terminal, characterized in that: The data processing terminal is provided with the following modules: An extraction module is used to extract frequency-domain periodic and non-periodic EEG features related to flow experience in EEG signals; A music player for presenting rhythmic music melodies; A touch screen for presenting visual cues that are phase-synchronized with the rhythmic music melody and tactile feedback formed by the user's tapping response; The sensorimotor synchronization analysis module is used to calculate the sensorimotor synchronization index between the user behavior and the sensory stimulation presentation rhythm; when the obtained sensorimotor synchronization index exceeds the preset interval, it is judged as poor synchronization, and the music melody of the music player is adjusted to stabilize the sensorimotor synchronization index within the preset interval; A heart flow calculation module is used to calculate the change of each EEG characteristic value relative to the initial EEG characteristic value after the sensory-motor synchronization index is stabilized; A feedback optimization module, when the change amount of each EEG characteristic value of the user reaches a preset standard, maintains the current music presentation rhythm of the music player; Otherwise, based on the starting music melody, the presentation rhythm of the sensory stimulation is changed within a reasonable range with a certain rhythm as the step length, and then the sensory-motor synchronization analysis module is used to perform sensory-motor synchronization analysis and the change amount of each EEG characteristic value is calculated through the heart flow calculation module.

Images