A training method and system based on brain-computer interface and virtual reality

Abstract

The present invention provides a training method based on brain-computer interface and virtual reality, and the method comprises: a test module in a patch detects fluctuations in electroencephalogram (EEG) data of a user, and determines whether a virtual reality device loaded with the patch is correctly worn; in response to detecting that the fluctuations in the electroencephalogram (EEG) data are normal, a collection module in the patch collects the electroencephalogram (EEG) data of the user in real time, and sends the electroencephalogram (EEG) data of the user collected by the collection module in real time to a calculation module; the calculation module inputs the collected electroencephalogram (EEG) data of the user into a course training model at a regular interval, outputs the training course, and sends it to a control end; the control end controls the virtual device to present the corresponding course until the calculation module detects that the collected electroencephalogram (EEG) data of the user is ideal EEG data, and the training is completed; the present invention combines deep learning and virtual reality, obtains the training screen required for electroencephalogram (EEG) data acquisition in real time, and the virtual reality presents the training screen to the user, and completes scientific guidance by matching the real-time electroencephalogram (EEG) data with the set training screen, so as to achieve scientific training and relaxation.

Description

Technical Field

[0001] The present invention relates to the field of rehabilitation training, and in particular to a training method and system based on brain-computer interface and virtual reality. Background Art

[0002] With the improvement of social level, people pay more and more attention to scientific training and scientific rest. For example, in yoga meditation, a specific EEG level must be reached to achieve scientific relaxation. How to scientifically and effectively implement training mode guidance under different EEG conditions is a problem that needs to be solved. Summary of the invention

[0003] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art, and propose a training method and system based on brain-computer interface and virtual reality, which combines deep learning and virtual reality to obtain the required training screens by acquiring EEG data in real time. Virtual reality presents the training screens to the user, and completes scientific guidance by matching real-time EEG data with the set training screens, achieving scientific training relaxation, and is easy for users to operate, simple and effective.

[0004] The technical solution adopted by the present invention is:

[0005] A training method based on brain-computer interface and virtual reality, comprising a control subsystem, wherein the control subsystem comprises a patch, a virtual device and a control terminal, wherein the patch comprises a test module, a collection module and a calculation module; the training method comprises:

[0006] The test module in the patch detects the fluctuation of the user's EEG data and determines whether the VR device with the patch installed is worn correctly; if the user's EEG data fluctuates abnormally, it will prompt "the patch is not installed or worn incorrectly";

[0007] In response to detecting that the EEG data fluctuation is normal, the acquisition module in the patch acquires the user's EEG data in real time, and sends the user's EEG data acquired in real time by the acquisition module to the calculation module;

[0008] The computing module inputs the collected user EEG data into the course training model at regular intervals, outputs the course, and sends it to the control end;

[0009] The control end controls the virtual device to present the corresponding course until the computing module detects that the collected user EEG data is ideal EEG data and the training is completed.

[0010] Furthermore, the normal EEG data fluctuation range is set in the test module;

[0011] When the acquired user's EEG data fluctuation is within the normal EEG data fluctuation range, the EEG data fluctuation is normal, and the acquisition module in the patch is triggered to collect the user's EEG data in real time;

[0012] When the obtained user's EEG data fluctuations are not within the normal EEG data fluctuation range, it will prompt "The patch is not installed or is worn incorrectly."

[0013] Furthermore, it also includes the training of the course training model, specifically:

[0014] Obtain the user's initial EEG data, the course, and the corresponding EEG data after the course is presented;

[0015] The user's initial EEG data and the EEG data corresponding to the course presentation are used as input data of the course training model, and the course is used as output data to train the course training model to obtain a trained course training model.

[0016] Furthermore, the course training models are LeNet-5, VGG, AlexNet, and MLP.

[0017] Another aspect of the embodiment of the present invention includes a training system based on brain-computer interface and virtual reality, including a control subsystem, the control subsystem includes a patch, a virtual device and a control terminal, the patch includes a test module, a collection module and a calculation module; the training system includes:

[0018] Detection unit: The test module in the patch detects the fluctuation of the user's EEG data and determines whether the VR device loaded with the patch is worn correctly; if the user's EEG data fluctuates abnormally, it will prompt "the patch is not installed or worn incorrectly";

[0019] Acquisition unit: In response to detecting that the EEG data fluctuation is normal, the acquisition module in the patch acquires the user's EEG data in real time, and sends the user's EEG data acquired in real time by the acquisition module to the calculation module;

[0020] Operation unit: The operation module inputs the collected user EEG data into the course training model at regular intervals, outputs the course, and sends it to the control end;

[0021] Control unit: The control end controls the virtual device to present the corresponding course until the computing module detects that the collected user EEG data is ideal EEG data and the training is completed.

[0022] Furthermore, the normal EEG data fluctuation range is set in the test module;

[0023] When the acquired user's EEG data fluctuation is within the normal EEG data fluctuation range, the EEG data fluctuation is normal, and the acquisition module in the patch is triggered to collect the user's EEG data in real time;

[0024] When the obtained user's EEG data fluctuations are not within the normal EEG data fluctuation range, it will prompt "The patch is not installed or is worn incorrectly."

[0025] Furthermore, it also includes the training of the course training model, specifically:

[0026] Obtain the user's initial EEG data, the course, and the corresponding EEG data after the course is presented;

[0027] The user's initial EEG data and the EEG data corresponding to the course presentation are used as input data of the course training model, and the course is used as output data to train the course training model to obtain a trained course training model.

[0028] Furthermore, the course training models are LeNet-5, VGG, AlexNet, and MLP.

[0029] On the other hand, the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned training steps based on brain-computer interface and virtual reality when executing the computer program.

[0030] Another aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned training steps based on brain-computer interface and virtual reality are implemented.

[0031] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention provides a training system based on brain-computer interface and virtual reality, and a training method based on brain-computer interface and virtual reality, comprising a control subsystem, wherein the control subsystem comprises a patch, a virtual device and a control terminal, wherein the patch comprises a test module, an acquisition module and a calculation module; the training method comprises: the test module in the patch detects fluctuations in the user's electroencephalogram data, and determines whether the virtual reality device loaded with the patch is correctly worn; if the user's electroencephalogram data fluctuations are abnormal, a prompt "the patch is not installed or is worn incorrectly" is given; in response to detecting that the electroencephalogram data fluctuations are normal, the acquisition module in the patch acquires the user's electroencephalogram data in real time, and transmits the user's electroencephalogram data acquired by the acquisition module in real time to the user. The collected EEG data is sent to the operation module; the operation module inputs the collected user EEG data into the course training model at a regular interval, outputs the training course, and sends it to the control end; the control end controls the virtual device to present the corresponding course until the operation module detects that the collected user EEG data is ideal EEG data, and the training is completed; the present invention proposes a training method and system based on brain-computer interface and virtual reality, which combines deep learning and virtual reality to obtain the required training screen of EEG data in real time, and virtual reality presents the training screen to the user, and completes scientific guidance by matching real-time EEG data with the set training screen, so as to achieve scientific training relaxation, which is easy for users to operate, simple and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A physical picture of a virtual reality device equipped with a patch provided by an embodiment of the present invention;

[0034] Figure 2 A flow chart of a training method based on brain-computer interface and virtual reality provided by an embodiment of the present invention;

[0035] Figure 3 A training drawing provided by an embodiment of the present invention;

[0036] Figure 4 A framework diagram of a training system based on brain-computer interface and virtual reality provided by an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0038] Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0039] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0040] The present invention proposes a training method based on brain-computer interface and virtual reality, which integrates breathing patterns of different needs and converts breathing data into real-time breathing images. Scientific guidance is completed by matching real-time breathing images with set scientific patterns. It is easy for users to operate, simple and effective.

[0041] like Figure 1 , is a real picture of a virtual reality device with a patch installed. The patch is a forehead patch, which is placed in the eye socket gasket of the VR device.

[0042] like Figure 2 , is a flow chart of a training method based on brain-computer interface and virtual reality. The specific method is as follows:

[0043] A training method based on brain-computer interface and virtual reality, comprising a control subsystem, wherein the control subsystem comprises a patch, a virtual device and a control terminal, wherein the patch comprises a test module, a collection module and a calculation module; the training method comprises:

[0044] S101: The test module in the patch detects the fluctuation of the user's EEG data and determines whether the virtual reality device loaded with the patch is worn correctly; if the user's EEG data fluctuates abnormally, it prompts "the patch is not installed or is worn incorrectly";

[0045] S102: In response to detecting that the EEG data fluctuation is normal, the acquisition module in the patch acquires the user's EEG data in real time, and sends the user's EEG data acquired by the acquisition module in real time to the calculation module;

[0046] Furthermore, the normal EEG data fluctuation range is set in the test module;

[0047] When the acquired user's EEG data fluctuation is within the normal EEG data fluctuation range, the EEG data fluctuation is normal, and the acquisition module in the patch is triggered to collect the user's EEG data in real time;

[0048] When the obtained user's EEG data fluctuations are not within the normal EEG data fluctuation range, it will prompt "The patch is not installed or is worn incorrectly."

[0049] S103: The operation module inputs the collected user EEG data into the trained course training model at a regular interval, outputs the course, and sends it to the control end;

[0050] It also includes the training of the course training model, specifically:

[0051] Obtain the user's initial EEG data, the course, and the corresponding EEG data after the course is presented;

[0052] The user's initial EEG data and the EEG data corresponding to the course presentation are used as input data of the course training model, and the course is used as output data to train the course training model to obtain a trained course training model.

[0053] The training models of the course are LeNet-5, VGG, AlexNet, and MLP.

[0054] S104: The control end controls the virtual device to present the corresponding course until the computing module detects that the collected user EEG data is ideal EEG data and the training is completed.

[0055] like Figure 3 , which is a training drawing provided by an embodiment of the present invention.

[0056] In specific scenarios, such as during yoga, meditation and relaxation content is provided to users for viewing, the relaxation part of the brain wave data is detected, and the values ​​of 1-100 are used to judge the user's relaxation state after watching the content in stages for adjustment. For example, in actual EEG acquisition, when the acquired value is between 0 and 40, the trained course training model outputs a slower-paced content to reduce the user's tension. When the acquired value is between 40 and 80, the trained course training model outputs the next standard content playback. When the acquired value is between 80 and 100, the trained course training model indicates that the user's current relaxation level exceeds expectations, and more advanced content can be provided for yoga practice; the adjusted EEG data of the entire process is used as feedback and input into the trained course training model, and the corresponding course training is output.

[0057] During meditation training, standard soothing content is provided for user experience. When the user's meditation value is between 1-60, it means that the user has not reached the standard of calmness. The trained course training model outputs a video to guide the user to scan themselves and enter the meditation state again. When the value is between 60-80, the trained course training model outputs the current standard soothing content to continue to guide the user into the state. When the detected meditation value is not between 80-100, the trained course training model output uses a pure natural environment to keep the user in a stable state of meditation.

[0058] like Figure 4 Another aspect of the present invention includes a training system based on brain-computer interface and virtual reality, including a control subsystem, wherein the control subsystem includes a patch, a virtual device and a control terminal, wherein the patch includes a test module, a collection module and a calculation module; the training system includes:

[0059] Detection unit 401: The test module in the patch detects the fluctuation of the user's EEG data and determines whether the virtual reality device loaded with the patch is worn correctly; if the user's EEG data fluctuates abnormally, it prompts "the patch is not installed or worn incorrectly";

[0060] Collection unit 402: In response to detecting that the EEG data fluctuation is normal, the collection module in the patch collects the user's EEG data in real time, and sends the user's EEG data collected in real time by the collection module to the calculation module;

[0061] Furthermore, the normal EEG data fluctuation range is set in the test module;

[0062] When the acquired user's EEG data fluctuation is within the normal EEG data fluctuation range, the EEG data fluctuation is normal, and the acquisition module in the patch is triggered to collect the user's EEG data in real time;

[0063] When the obtained user's EEG data fluctuations are not within the normal EEG data fluctuation range, it will prompt "The patch is not installed or is worn incorrectly."

[0064] Operation unit 403: The operation module inputs the collected user EEG data into the course training model at a regular interval, outputs the course, and sends it to the control end;

[0065] Furthermore, it also includes the training of the course training model, specifically:

[0066] Obtain the user's initial EEG data, the course, and the corresponding EEG data after the course is presented;

[0067] The user's initial EEG data and the EEG data corresponding to the course presentation are used as input data of the course training model, and the course is used as output data to train the course training model to obtain a trained course training model.

[0068] Furthermore, the course training models are LeNet-5, VGG, AlexNet, and MLP.

[0069] Control unit 404: The control end controls the virtual device to present the corresponding course until the computing module detects that the collected user EEG data is ideal EEG data and the training is completed.

[0070] Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 520 and executable on the processor 520. When the processor 520 executes the computer program 511, a training method based on brain-computer interface and virtual reality provided by an embodiment of the present invention is implemented.

[0071] Since the electronic device introduced in this embodiment is the device used to implement the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, the technical personnel in this field can understand the specific implementation mode of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention is not introduced in detail here. As long as the equipment used by the technical personnel in this field to implement the method in the embodiment of the present invention falls within the scope of protection of the present invention.

[0072] See also Figure 6 , Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0073] like Figure 6 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 611 is stored. When the computer program 611 is executed by a processor, a training method based on brain-computer interface and virtual reality provided by an embodiment of the present invention is implemented.

[0074] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] The present invention provides a training system based on brain-computer interface and virtual reality, and a training method based on brain-computer interface and virtual reality, comprising a control subsystem, wherein the control subsystem comprises a patch, a virtual device and a control terminal, wherein the patch comprises a test module, an acquisition module and a calculation module; the training method comprises: the test module in the patch detects fluctuations in the user's electroencephalogram data, and determines whether the virtual reality device loaded with the patch is correctly worn; if the user's electroencephalogram data fluctuations are abnormal, a prompt "the patch is not installed or is worn incorrectly" is given; in response to detecting that the electroencephalogram data fluctuations are normal, the acquisition module in the patch acquires the user's electroencephalogram data in real time, and transmits the user's electroencephalogram data acquired in real time by the acquisition module to the user The collected EEG data is sent to the operation module; the operation module inputs the collected user EEG data into the course training model at a regular interval, outputs the training course, and sends it to the control end; the control end controls the virtual device to present the corresponding course until the operation module detects that the collected user EEG data is ideal EEG data, and the training is completed; the present invention proposes a training method and system based on brain-computer interface and virtual reality, which combines deep learning and virtual reality to obtain the required training screen of EEG data in real time, and virtual reality presents the training screen to the user, and completes scientific guidance by matching real-time EEG data with the set training screen, so as to achieve scientific training relaxation, which is easy for users to operate, simple and effective.

[0077] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. The various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

[0078] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A training system based on brain-computer interface and virtual reality, characterized in that: The training system comprises a control subsystem, wherein the control subsystem comprises a patch, a virtual device and a control terminal, wherein the patch comprises a test module, a collection module and a calculation module; and the training system comprises: Detection unit: The test module in the patch detects the fluctuation of the user's EEG data and determines whether the VR device with the patch installed is worn correctly; if the user's EEG data fluctuates abnormally, it will prompt "the patch is not installed or worn incorrectly"; Acquisition unit: In response to detecting that the EEG data fluctuation is normal, the acquisition module in the patch acquires the user's EEG data in real time, and sends the user's EEG data acquired in real time by the acquisition module to the calculation module; Operation unit: The operation module inputs the collected user EEG data into the trained course training model at regular intervals, outputs the course, and sends it to the control end; Control unit: The control end controls the virtual device to present the corresponding course until the computing module detects that the collected user EEG data is ideal EEG data and the training is completed; The normal EEG data fluctuation range is set in the test module; When the acquired user's EEG data fluctuation is within the normal EEG data fluctuation range, the EEG data fluctuation is normal, and the acquisition module in the patch is triggered to collect the user's EEG data in real time; When the obtained EEG data fluctuation of the user is not within the normal EEG data fluctuation range, it will prompt "The patch is not installed or is worn incorrectly"; The system also includes the training of the course training model, specifically: Obtain the user's initial EEG data, the course, and the corresponding EEG data after the course is presented; The user's initial EEG data and the EEG data corresponding to the course presentation are used as input data of the course training model, and the course is used as output data to train the course training model to obtain a trained course training model; The training models of the course are LeNet-5, VGG, AlexNet, and MLP; The system is applied in the yoga process to provide meditation and relaxation content for users to watch, detect relaxation data in brain wave data, and use values ​​1-100 to judge the user's relaxation state after watching the content in stages for adjustment. In actual EEG collection, when the acquired value is between 0 and 40, the trained course training model outputs slow-paced content; when the acquired value is between 40 and 80, the trained course training model outputs and keeps entering the next standard content playback; when the acquired value is between 80 and 100, the trained course training model provides advanced content for yoga practice; the adjusted EEG data of the entire process is used as feedback and input into the trained course training model to output the corresponding course training.

Images