Method for controlling airwave gloves based on brain wave signals and helmet equipment

Abstract

The invention provides a method for controlling airwave gloves based on brain wave signals and helmet equipment. In the method, a controller of helmet equipment receives electroencephalogram signal output data sent by the helmet equipment based on a TGAM chip; analyzing the electroencephalogram signal output data to obtain original electroencephalogram signal data; electroencephalogram signal features are determined according to the original electroencephalogram signal data; inputting the electroencephalogram signal characteristics into a preset electroencephalogram signal decoding model, and determining an intention control instruction of the user to the airwave glove; and controlling the airwave glove to execute corresponding operation actions according to the intention control instruction. By implementing the technical scheme provided by the invention, accurate control on the airwave glove based on the brain wave signal can be realized, so that a more natural and efficient interaction mode is provided for a user.

Description

Technical Field

[0001] The present application relates to the field of brain-computer interface technology, and in particular to a method for controlling air wave gloves based on brain wave signals and a helmet device. Background Art

[0002] With the development of smart wearable devices, it has become possible to monitor brain wave signals through helmets. At the same time, air wave gloves, as a new type of human-computer interaction device, have broad application prospects in rehabilitation, games and other fields.

[0003] Traditional air wave gloves mainly rely on input devices such as physical buttons or touch screens for control, and the switch and output direction of the air wave gloves are controlled by manually clicking buttons or touching the screen.

[0004] However, it is difficult to achieve continuous and natural control in the traditional method of controlling air wave gloves, and there is no direct correspondence between the physical buttons and the output of the air wave gloves. Therefore, it is difficult for the existing technology to control the air wave gloves as intuitively as directly controlling the palm. Summary of the invention

[0005] The present application provides a method for controlling air wave gloves based on brain wave signals, which is used to achieve seamless connection between brain wave signals and glove control in situations where it is difficult to accurately and directly control air wave gloves in the prior art, thereby improving the accuracy and efficiency of controlling air wave gloves.

[0006] In the first aspect, the present application provides a method for controlling air wave gloves based on brain wave signals, which is applied to a helmet device, and the method includes: receiving EEG signal output data sent by a helmet device based on a TGAM chip; parsing the EEG signal output data to obtain original EEG signal data; determining EEG signal characteristics based on the original EEG signal data; inputting the EEG signal characteristics into a preset EEG signal decoding model to determine the user's intended control instructions for the air wave gloves; and controlling the air wave gloves to perform corresponding operation actions according to the intended control instructions.

[0007] By adopting the above technical solution, it is possible to achieve precise control of air wave gloves based on brain wave signals, realize seamless connection between brain wave signals and glove operations, improve the accuracy and convenience of control, and provide users with a more natural and efficient way of interaction.

[0008] In combination with some embodiments of the first aspect, in some embodiments, before the step of inputting the EEG signal feature into a preset EEG signal decoding model to determine the user's intended control instructions for the air wave gloves, it also includes: after determining that the helmet device is turned on by the user and connected to Bluetooth, light is detected on one side of the Bluetooth connection device, then it is determined that the helmet device has established a communication connection with the air wave gloves; the noise value is determined by debugging the device; if the noise value is detected to change from 0 to 100, it is determined that the user has poor wearing contact, and the speaker is controlled to issue a re-wearing prompt message; if the noise value is displayed as changing from full to 0 when the user uses the helmet device, it indicates that the user is wearing it correctly.

[0009] By adopting the above technical solution, a stable communication connection between the helmet device and the air wave gloves can be ensured, and problems with poor contact when the user is wearing the helmet device can be discovered in a timely manner, guiding the user to wear the helmet device correctly, thereby improving the stability and reliability of the device use and ensuring that the user can use brain wave signals to control the gloves normally.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of receiving the EEG signal output data sent by the helmet device based on the TGAM chip, it also includes: detecting whether the checksum of the small packet data in the EEG signal output data is correct; if correct, determining the original EEG signal data; if incorrect, discarding the corresponding data packet; after determining the packet loss rate, if the packet loss rate is lower than the set threshold, controlling the alarm device to issue an alarm message.

[0011] By adopting the above technical solution, the accuracy and integrity of the EEG signal output data can be guaranteed. When the packet loss rate is lower than the set threshold, the control alarm device sends an alarm message to remind the user to pay attention to data transmission problems, thereby improving the stability and reliability of the system and providing a reliable data basis for accurately controlling the air wave gloves.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of parsing the EEG signal output data to obtain the original EEG signal data, it also includes: extracting and screening the original EEG signal data to determine the user's effective concentration data and relaxation data; if the effective concentration data and the relaxation data control the strength of the air wave gloves.

[0013] By adopting the above technical solution, the strength of the air wave gloves can be accurately controlled according to the user's effective concentration data and relaxation data. The original EEG signal data is extracted and screened to obtain the user's effective concentration data and relaxation data, which can reflect the user's intention and state. The strength of the gloves is controlled according to these data, making the operation of the gloves more in line with the user's needs, improving the user experience, and realizing more personalized and intelligent control.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the step of controlling the air wave glove to perform a corresponding operation action according to the intention control instruction specifically includes: determining the component to be executed of the air wave glove according to the intention control instruction; determining the required action type corresponding to the component to be executed; after determining the air wave glove control instruction according to the action type and the component to be executed, sending the air wave glove control instruction to the air wave glove, so that the air wave glove controls the pressure and flow of the component to be executed to complete the action type.

[0015] By adopting the above technical solution, the components to be executed and the required action type of the glove are determined according to the intended control instruction, and then the control instruction is determined and sent to the glove, so that the glove controls the pressure and flow of the components to be executed to complete the action type. This ensures that the operation of the glove is accurate, improves the accuracy and reliability of the control, and meets the user's needs for different operation actions.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of controlling the air wave glove to perform a corresponding operation action according to the intention control instruction, it also includes: continuing to obtain the original EEG signal data, and extracting EEG signal features in the original EEG signal data; determining the user's satisfaction with the action performed by the air wave glove based on the extracted EEG signal features; if the EEG signal features indicate that the user is not satisfied with the current execution result, re-acquiring the user's new EEG signal features; and updating the intention control instruction based on the new EEG signal features and sending it to the air wave glove.

[0017] By adopting the above technical solution, after the control glove performs the operation, the original EEG signal data is continuously obtained, the EEG signal features are extracted, and the user's satisfaction with the executed action is judged. If the user is not satisfied, the new EEG signal features are re-acquired, and the intention control instructions are updated and sent to the glove, thereby realizing real-time adjustment of the glove's execution action and improving user satisfaction and usage experience.

[0018] In combination with some embodiments of the first aspect, in some embodiments, before the step of determining the EEG signal features based on the original EEG signal data, it also includes: collecting original EEG signal sample data of multiple users, and training to obtain a general domain EEG signal feature recognition model; when receiving information that a new user uses the helmet device, collecting new original EEG signal data of the new user; using the general domain brain wave signal recognition model as a pre-trained model, and adjusting the pre-trained model according to the new original EEG signal data to obtain a user-personalized EEG signal decoding model suitable for the new user.

[0019] By adopting the above technical solution, the original EEG signal sample data of multiple users are collected and trained to obtain the general domain EEG signal feature recognition model. When a new user uses the helmet device, the new original EEG signal data is collected and adjusted with the general domain model as the pre-trained model to obtain a personalized model suitable for the new user. This can improve the accuracy and adaptability of EEG signal decoding, better meet the needs of different users, and improve the performance and user experience of the entire system.

[0020] In a second aspect, the present application provides a helmet device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the helmet device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, the present application provides a computer program product comprising instructions, which, when executed on a helmet device, enables the helmet device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a helmet device, causes the helmet device to execute the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The invention adopts a technical method of receiving the EEG signal output data sent by the helmet device based on the TGAM chip, analyzing it, determining the EEG signal characteristics, and inputting the EEG signal characteristics into a preset EEG signal decoding model to determine the intended control instructions. Therefore, the problem that the air wave gloves in the prior art rely on input devices such as physical buttons or touch screens for control and are difficult to achieve continuous and natural control is effectively solved, thereby achieving the technical effect of directly and naturally controlling the air wave gloves through brain wave signals.

[0024] 2. The invention adopts a technical method of determining the parts to be executed and the required action type of the air wave gloves according to the intended control instructions, and then determining the control instructions according to the action type and the parts to be executed, and sending them to the air wave gloves to control the pressure and flow of the parts to be executed to complete the action type. Therefore, the problem in the prior art that it is difficult to control the air wave gloves intuitively like directly controlling the palm of the hand is effectively solved, thereby achieving the technical effect of accurately and intuitively controlling the air wave gloves to perform corresponding operation actions.

[0025] 3. The invention adopts the technical means of continuing to obtain the original EEG signal data, extracting the EEG signal features therein, and judging the user's satisfaction with the execution of the air wave gloves based on the EEG signal features. If the user is not satisfied, the new EEG signal features are re-acquired and the intention control instructions are updated and sent to the air wave gloves. Therefore, the problem that the air wave gloves cannot adjust the execution actions in time according to user feedback in the prior art is effectively solved, thereby achieving the technical effect of adjusting the execution actions of the air wave gloves in real time according to user feedback and improving user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a method for controlling air wave gloves based on brain wave signals in an embodiment of the present application; Figure 2 is another flow chart of the method for controlling air wave gloves based on brain wave signals in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of a physical device of the helmet device in the embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] The method of controlling air wave gloves based on brain wave signals in the embodiment of the present application is adopted, and the EEG signal output data sent by the helmet device based on the TGAM chip is received, parsed and processed to achieve precise control of the air wave gloves, which can not only achieve a more natural and intuitive interaction method, but also meet the needs of users in different scenarios. The following describes the scenario in which the method of controlling air wave gloves based on brain wave signals in the present application is used. For example, in the rehabilitation training scenario, the patient wears a helmet device and an air wave glove, and the helmet device collects the patient's EEG signal and sends it to the controller. The controller receives the EEG signal output data, and after parsing and processing, determines the patient's intended control instructions for the air wave gloves, such as fisting, stretching, etc. The air wave gloves perform corresponding operation actions according to the instructions to help the patient perform hand rehabilitation training. This control method based on brain wave signals can respond to the patient's intention more naturally and improve the effect of rehabilitation training. For example, in the game entertainment scenario, the player wears a helmet device and an air wave glove to control the character actions in the game through brain wave signals. Players can trigger different action instructions by concentrating or relaxing, making the game experience more immersive and interesting.

[0030] It can be seen that the method of controlling air wave gloves based on brain wave signals in the embodiment of the present application can not only achieve precise control of the air wave gloves, but also bring users a more convenient and natural interactive experience, expanding the application prospects of air wave gloves in rehabilitation, games and other fields.

[0031] For ease of understanding, the following is a description of the process of the method provided by this implementation. Figure 1 , which is a flow chart of a method for controlling air wave gloves based on brain wave signals in an embodiment of the present application.

[0032] S101, receiving EEG signal output data sent by a helmet device based on a TGAM chip; The helmet device integrates a TGAM chip, which can output EEG signal data at a rate of about 513 packets per second, which contains two types of data packets, small packets and large packets. The format of the small packet is AA AA 04 80 02xxHigh xxLow xxCheckSum. Among them, AA AA 04 80 02 is the packet header, which is fixed. The two bytes xxHigh and xxLow constitute the raw EEG signal data rawdata, and xxCheckSum is the checksum. So each small packet contains a raw EEG signal data. The helmet device outputs about 512 such small packets per second, that is, 512 raw EEG signal data.

[0033] The format of the large packet is AA AA 08 80 02 xxHigh xxLow xxHigh xxLow ......xxCheckSum. The large packet contains multiple raw EEG signal data, and each EEG signal data is also represented by xxHigh and xxLow. The number of large packets is less than that of small packets. The helmet controller needs to be equipped with a Bluetooth or USB interface. After establishing a connection with the helmet, it receives the EEG signal data according to the specific transmission protocol of the TGAM chip. Whether it is a small packet or a large packet, the controller needs to first check the checksum to verify the data integrity. The specific verification method is as follows: First, the server will detect whether the checksum in each EEG signal data packet is correct. The checksum is a checksum calculated according to a fixed algorithm when the data packet is generated. The server can use the same algorithm to recalculate it once and compare the checksum value. If the calculated checksum is consistent with the checksum value in the data packet, it means that there is no error in the transmission of the data packet and the verification is successful. The server will continue to process the data packet and extract the raw EEG signal data from it. If the calculated checksum value is inconsistent with the data packet value, it means that a data error occurred during the transmission and cannot be guaranteed to be intact. At this time, the server will directly discard the data packet and no longer process it.

[0034] The server will then count the total number of data packets and the number of discarded damaged data packets, and calculate the packet loss rate. If the packet loss rate is lower than the preset tolerable threshold, it means that the EEG signal data transmission is still within the controllable range. If the packet loss rate is higher than the threshold, it means that the transmission quality is already relatively poor and there are too many packet losses, which will directly affect the subsequent signal analysis and motion control. When the packet loss rate exceeds the threshold, the server will send an alarm to the user, prompting that there is a problem with the helmet signal transmission and the helmet device needs to be checked. In this way, by checking and verifying the data packets, the erroneous data can be filtered out to avoid interference with the signal analysis; and the user can be prompted with the quality problem of the helmet signal through packet loss rate monitoring to ensure that the server receives complete and reliable original EEG signal data for subsequent precise control.

[0035] In some embodiments, before receiving the original EEG signal data, the server side needs to ensure that a normal communication connection is established between the helmet device and the air wave glove, and that the helmet device is correctly worn by the user. Specifically, after determining that the user turns on the helmet device and connects to the server via Bluetooth, the server side will detect whether a light indication indicating a successful connection is issued on the Bluetooth connection device. If the Bluetooth connection device has shown that the connection is normal, it can be determined that the communication link between the helmet device and the air wave glove has been established. Secondly, the server side needs to monitor the noise value of the helmet device when collecting EEG signals in real time to determine whether the user is wearing it correctly. The server can request the debugging module in the helmet device to return a real-time noise monitoring value, which ranges from 0 to 100. If the noise value is detected to vary between 0 and 100, it means that the contact between the helmet and the user is unstable, and there is movement or falling off. At this time, the server side will control the speaker in the helmet to issue a voice prompt of "Please adjust the helmet position" to let the user put on the helmet again. If the noise value is stably displayed at 0, it means that the helmet is in good contact with the scalp and there is no problem of unstable contact. The server side determines that the user has worn the helmet correctly and can start to collect EEG signal data normally. In addition, the server will continue to monitor the noise level during wearing. If the noise level gradually increases from 0, the user will be prompted that the helmet contact has failed and needs to be worn again.

[0036] In this way, by monitoring the Bluetooth connection status and noise value, the server can detect the connection status between the helmet device and the air wave gloves, as well as the user's wearing status of the helmet device in real time, ensuring that the original EEG signal data is collected under normal communication and wearing conditions to avoid data loss or errors.

[0037] In some embodiments, before extracting features and parsing intent from the original EEG signal, the server can first establish a reliable general domain EEG signal feature recognition model, and then use the new user data for personalized adjustment. First, the server collects the original EEG signal sample data generated by multiple users wearing helmet devices. These sample data need to cover a variety of different groups of people and be labeled with corresponding action intention labels. Then, the server uses the collected massive labeled training data to train a general domain EEG signal feature recognition model using a deep learning neural network algorithm. The model gradually optimizes parameters through supervised learning, so that the network can accurately extract features from the original EEG signal and identify the corresponding intended actions. The general model will learn the generally applicable EEG signal analysis rules. However, due to individual differences, it is impossible to be completely applicable to every new user. Therefore, when a new user starts using the system, the server will first collect the new original EEG signal data generated by the user wearing a helmet, and these data will be used as a fine-tuning data set. Then, the server uses the trained general domain model as a pre-trained model and loads its weight parameters. Taking the pre-trained model as the starting point, the model is fine-tuned using the new user data, and the model parameters are adjusted through continued training to make it more suitable for the user's data distribution. After fine-tuning with new user data, the server obtains a user-specific EEG signal feature recognition and decoding model. This personalized model can more accurately analyze the user's EEG signals and decode intent for subsequent precise control. This method of general model pre-training plus personalized fine-tuning can better adapt to the differences between different users and improve the effect of EEG signal analysis and intent recognition.

[0038] S102, analyzing the EEG signal output data to obtain original EEG signal data; After receiving the EEG signal data output by the TGAM chip, the controller needs to parse and process it to obtain the original EEG signal data. The raw data can be parsed from the small package by setting: rawdata = (xxHigh<<8) | xxLow;if(rawdata>32768){ rawdata =65536;}, and the signal strength, concentration, relaxation and other information can also be obtained from the large package. After obtaining effective concentration and relaxation data, the server can adjust the strength parameters of the air wave gloves in real time according to these data. For example, if the concentration data is 80 and the relaxation data is 20, it means that the user is currently concentrating on controlling the gloves, and the server will increase the grip of the gloves accordingly. If the concentration data is 20 and the relaxation data is 80, the grip is reduced to make the control softer. By analyzing the original EEG signal and extracting the user status feature data, the server can achieve precise adjustment of the strength of the air wave gloves to adapt to the user's real-time state. When the user's attention is distracted, the strength can be automatically reduced; when the user is focused, the strength is increased to produce a clear action. This not only ensures the safety of glove control, but also improves the comfort of use.

[0039] S103, determining EEG signal features according to the original EEG signal data; After obtaining the user's original EEG signal data, the server needs to analyze the data features to extract effective information. The server will collect a large number of original EEG signal data samples generated by the user and perform preprocessing to remove invalid components such as noise. Then, the server will extract the features of the EEG signal from both the time domain and the frequency domain.

[0040] In the time domain, the server analyzes the waveform characteristics of the EEG signal curve, such as peak value, peak-to-valley value, period, frequency, amplitude and other parameters. For example, methods such as wavelet transform can be used to analyze the value of the signal at different times to determine the different states of the brain wave, such as lambda wave, alpha wave, beta wave, etc. In the frequency domain, the server will perform a fast Fourier transform to obtain the power distribution of the EEG signal at different frequencies. According to the concentration of energy at a specific frequency, the user's concentration, relaxation state, etc. can be judged. The dynamic changes of the signal spectrum of the user under different cognitive tasks can also be analyzed. Combining the time domain and frequency domain features, the server can extract EEG signal features that represent the user's intention, emotion and cognitive state.

[0041] S104, inputting the EEG signal feature into a preset EEG signal decoding model to determine the user's intended control instruction for the air wave glove; The server side uses the extracted EEG signal features as input to a pre-trained EEG signal decoding model. The EEG signal decoding model can be constructed using a deep learning method. The EEG signal decoding model includes an input layer, a hidden layer, and an output layer. The input layer receives EEG signal features, the hidden layer extracts and converts features, and the output layer predicts the intended control instructions for the air wave glove.

[0042] The server can collect a large amount of EEG signal data when users perform various gestures or actions, label them, and build a training set. Then, the model is trained using the training set through a machine learning algorithm. Common model training methods include supervised learning and transfer learning. Supervised learning is to directly train the model using labeled training data. Transfer learning, on the other hand, uses a pre-trained model in a related field as the initial model, and then uses the user's data for fine-tuning. The training process continuously optimizes the loss function so that the model output accurately fits the label corresponding to the training data.

[0043] The trained EEG signal decoding model can classify or predict the input EEG signal features. For example, it can predict whether the user wants to perform actions such as "clenching a fist" or "opening hands". The server collects the EEG signal data generated when the user wears the device, extracts the signal features and inputs them into the model to obtain the prediction results of the intended control instructions. If the model determines that the user wants to perform the "clenching fist" action, the output control instruction is "clenching a fist". The server can also perform time series analysis based on EEG signals of multiple time segments to determine the duration of the control instruction.

[0044] In addition, the server can further optimize the model and achieve personalized adaptation by continuously collecting EEG signal data generated by users and feedback on the execution results of the air wave gloves. The accuracy of prediction can be improved by using the transfer learning method to fine-tune the parameters of the model using the user's own data. The server can build an independent personalized model for each user, or use joint training to obtain a general model. The model needs to be continuously updated and iterated with new data to make accurate judgments on the user's intentions.

[0045] S105: Control the air wave glove to perform corresponding operation actions according to the intention control instruction.

[0046] After obtaining the intended control instruction for the air wave glove, the server sends the instruction to the air wave glove to drive it to perform the corresponding operation action.

[0047] For example, if the recognition result is "clenching a fist", the server generates a corresponding instruction and sends it to the glove to control the glove to drive the fist-clenching action. The control instruction sent by the server contains specific parameter information, including the finger parts of the air wave glove to be controlled, the air pressure or airflow to be applied to each part, the duration of the action and other parameters. The server first parses the intention control instruction to determine the parts to be executed of the air wave glove, usually each part of the finger, and then calculates the air pressure and airflow direction required to be generated for each part. Controlling the fist-clenching action requires coordinating the magnitude and direction of the force of each finger. First, the server determines the parts to be executed of the air wave glove according to the intention control instruction. The air wave glove contains multiple finger chambers, and the server needs to clearly control all fingers or specific fingers, such as only controlling the index finger and thumb. Then, the server determines the required action type corresponding to the part to be executed. The action type can be different gestures, such as clenching a fist, making a gesture, etc. It can also be a more detailed single finger action, such as curling the index finger, pressing the thumb down, etc. The server will refer to the preset gesture action definition to find the action type that matches the intention control instruction.

[0048] After clarifying the action type and the involved finger parts, the server can generate specific control instructions for the air wave gloves. The control instructions will contain the working parameters of each finger chamber, such as air pressure, airflow direction, working time, etc. Controlling different types of finger actions requires adjusting the parameters of each chamber. For example, for the action type of "clenching a fist", the server will control all finger chambers to generate inward airflow and gradually increase the air pressure to bend and clench the fingers. For the "OK" gesture, only the thumb, index finger, and middle finger chambers may be controlled to keep them upright or bent into an OK shape. The server can adjust the specific parameters of each chamber according to the preset gesture model.

[0049] The server then encapsulates the detailed parameter information into a data packet of control instructions and sends it to the air wave glove via WIFI or Bluetooth connection. After the air wave glove receives the instruction, its own hardware system can parse the instruction data packet and drive the air pump system to provide airflow with the correct pressure to different finger chambers to make the palm form a fist. During the execution of the instruction, the server and the glove need to maintain a connection to transmit control parameters and status data. In addition to the fisting action, the control instructions sent by the server can also drive the glove to perform various gestures, such as grabbing, clicking, rotating and other complex actions. When generating control parameters, the server can refer to the preset gesture model library or the recorded user habit data. The hand shape and usage habits of different users can also be adapted through personalized models. After sending the instruction, the server can also evaluate the control effect based on the execution status returned by the air wave glove and provide feedback for model adjustment.

[0050] The method of controlling air wave gloves based on brain wave signals in the embodiment of the present application is adopted, by receiving the original EEG signal data, analyzing and obtaining features, inputting the decoding model to determine the control instructions, and controlling the air wave gloves to perform actions according to the instructions, thereby realizing precise and natural control of the air wave gloves based on brain wave signals, which not only realizes the seamless connection between brain wave signals and glove control, but also improves the accuracy and efficiency of control, bringing better interactive experience to users.

[0051] In some embodiments, after controlling the air wave gloves to perform corresponding operation actions according to the intention control instructions, the server can continue to collect the original EEG signal data of the user during use, perform feature extraction, and analyze the user's satisfaction with the current execution results, so as to adjust the subsequent air wave gloves execution actions.

[0052] After combining the above scenarios, the following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the method for controlling air wave gloves based on brain wave signals in an embodiment of the present application.

[0053] S201, continue to acquire the original EEG signal data, and extract EEG signal features from the original EEG signal data; After controlling the air wave gloves to perform the operation, the server will continue to collect the original EEG signal data generated by the user through the connection with the helmet device worn by the user. These original EEG signal data reflect the user's judgment and feelings on the results of the air wave gloves performing the operation. The server will obtain the original EEG signal data generated by the user in a certain period of time in real time. Then, the server preprocesses the collected original EEG signal data, including removing invalid signals, reducing noise, etc., to obtain the effective original EEG signal data after cleaning. Next, the server extracts time domain features and frequency domain features from the preprocessed original EEG signal data. For example, the peak value, valley value, cycle, frequency and other parameters of the EEG signal are analyzed in the time domain. These features reflect the user's judgment on the effect of the action execution. Finally, the server will obtain the EEG signal features of the user's operation results on the air wave gloves based on the extracted time domain features and frequency domain features. These features will be used to judge the user's satisfaction in the next step.

[0054] S202, determining the user's satisfaction with the action performed by the air wave glove according to the extracted EEG signal features; After obtaining the EEG signal features of the user's operation results on the air wave gloves, the server inputs these feature data into the trained EEG signal satisfaction evaluation model to judge the user's satisfaction. The satisfaction evaluation model can be a regression model, which inputs the user's EEG signal features and outputs a satisfaction score, indicating the user's satisfaction with the effect of the current glove performing the action. The training process of the satisfaction evaluation model is as follows: the server needs to collect the original EEG signal data of multiple users using the air wave gloves. After the user uses the gloves to perform different actions, the user is prompted to make a subjective score. The score can be a number of 1-100 points indicating the degree of satisfaction, or provide two types of options: satisfied and dissatisfied. At the same time, the user's original EEG signal data before and after the score is collected. Establish a relationship between the data sample and the corresponding satisfaction label, and repeat the process to collect EEG signal data of a large number of users, multiple actions and environments. Ensure that the number of data samples is sufficient and includes various situations. Preprocess the collected original EEG signal data to obtain a digital feature vector representing the EEG signal. The extracted EEG signal features and the corresponding satisfaction score labels are combined into a data set for model training. The training set data set is used for supervised learning, and the model parameters are continuously optimized to make the satisfaction score predicted by the model as consistent as possible with the actual label. The validation set data is used to evaluate the model's predictive performance, such as accuracy, mean square error, etc., and the model is further improved based on the evaluation results.

[0055] The server inputs the user's EEG signal features obtained in real time into the model, and the model can output a satisfaction score from 0 to 100. The server sets a satisfaction threshold, such as a score below 60, which means the user is dissatisfied. Satisfaction evaluation can also be a classification model that divides satisfaction into two categories, inputs EEG signal features, and outputs a binary classification result of satisfaction or not. The training of the classification model can be supervised by explicit feedback from users to obtain a sample data set of satisfaction and dissatisfaction. New features are input into the model, and the classification of whether the user is satisfied with the current execution effect is output.

[0056] In addition to relying on modeling, the server can also directly judge satisfaction by analyzing the data of EEG signals in specific frequency bands. For example, if the beta wave is abnormal, it means that the user is dissatisfied. Combining the model with direct signal analysis, the server can accurately judge the user's satisfaction with the actions performed by the glove.

[0057] S203, if the EEG signal feature indicates that the user is not satisfied with the current result of the action being performed, then reacquire a new EEG signal feature of the user; If the server determines that the user is not satisfied with the current execution effect of the air wave gloves based on the user's EEG signal characteristics, it is necessary to re-collect the user's EEG signal data to obtain the user's new control intention. After determining that the user is dissatisfied, the server will specify a time period to continue to obtain the original EEG signal data generated when the user wears the helmet device. Repeat the above steps S101 to S105, perform preprocessing, feature extraction and other operations on the collected new data, and obtain EEG signal characteristics representing the user's new control intention. These newly collected EEG signal characteristics can better express the user's new requirements for the gloves to perform actions, such as the need to clench the fist faster or change the strength. Acquiring new EEG signal characteristics is conducive to the server generating updated control instructions to meet the user's new intentions. If the user continues to show dissatisfaction, the server can collect EEG signal characteristics multiple times, or prompt the user to give clear feedback, and directly inform the server of its new control intention requirements through buttons or voice, so as to generate more accurate control instructions.

[0058] By reacquiring the user's EEG signal characteristics, the server can dynamically adjust instructions to make the operation of the air wave gloves more in line with the user's intention, thereby improving user satisfaction.

[0059] S204, updating the intention control instruction according to the new EEG signal feature and sending it to the air wave glove.

[0060] If the server determines that the user is not satisfied with the effect of the air wave gloves, it will re-collect the user's EEG signal data and obtain new EEG signal features to indicate the user's new control intention. The server will use the new EEG signal features and re-input them into the trained EEG signal decoding model, and the model will output the updated intention control instruction results. For example, the user's original intention was to control the air wave gloves to perform a "fist" action, but the force was too small and the user was not satisfied. After the server re-acquires the EEG signal features, the EEG signal decoding model will output the updated intention control instruction "clench your fist", indicating that the force needs to be increased.

[0061] After obtaining the new intention control instruction, the server will update the control parameters of the generated air wave gloves accordingly. For example, increase the air pressure value of each finger chamber and adjust the airflow direction to achieve greater fist strength. The server encapsulates the instruction data containing detailed control parameters and sends it to the air wave gloves via Bluetooth connection. After receiving the new control instruction data packet, the air wave gloves will re-parse the instruction parameters and adjust the pressure output of the air pump system accordingly. Finally, drive different finger chambers to generate adjusted airflow to implement the new intention control instruction transmitted by the server. After the air wave gloves execute the action corresponding to the new control instruction, the server will continue to collect the user's EEG signal feedback. If the user is still not satisfied, further iterative updates can be triggered until the user is satisfied. This closed-loop control method can continuously optimize the execution effect of the air wave gloves and provide a more accurate and comfortable interactive experience.

[0062] In some embodiments, after obtaining the EEG signal characteristics of the user, the server can also match it with the preset emotional standard brain wave signal to determine the current emotional state of the user. The server will pre-store the standard EEG signal characteristics of different emotional states in the database, such as standard brain wave signal patterns in states such as joy, pressure, and relaxation. This can be obtained by collecting a large amount of EEG signal data with clear emotional labels for training. After obtaining the EEG signal characteristics of the current user, the server will match and compare it with the samples in the standard brain wave signal library. The similarity between the feature vectors can be calculated. If the similarity between the current user's brain wave signal and a standard emotional waveform is higher than the set threshold, the user is judged to be in the current emotional state. For example, if the similarity between the currently collected user's brain wave signal and the preset "pressure" standard brain wave is very high, the user is judged to be in a stress state. If the similarity with the "relaxation" standard brain wave is high, the user is judged to be in a relaxed state.

[0063] After obtaining the user's current emotion, the server will comprehensively consider the recognized intention control instructions and the current emotion to determine the control parameter adjustment instructions sent to the air wave gloves. For example, if the user's intention instruction is "clench your fist", but the current emotion is "stress", the server will adjust the strength control parameters accordingly, and the generated control instruction is "lightly clench your fist", and the lightly clenched strength control parameters will be sent to the air wave gloves. If the current emotion is "relaxation", the control parameters of "normal fist clenching" are generated. By combining user emotions and intention instructions to adjust the control parameters, the actions of the air wave gloves can be made more in line with the user's current overall state, and the operation can be smoother and more comfortable.

[0064] The following describes the helmet device in the embodiment of the present invention from the perspective of hardware processing. Figure 3 , is a schematic diagram of a physical device structure of a helmet device in an embodiment of the present application.

[0065] It should be noted that Figure 3 The structure of the helmet device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0066] like Figure 3 As shown, the helmet device includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0067] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0068] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0069] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0071] Specifically, the helmet device of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the method of controlling air wave gloves based on brain wave signals provided in the above embodiment is implemented.

[0072] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the helmet device described in the above embodiment; or may exist independently without being assembled into the server. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the server, the server implements the method for controlling air wave gloves based on brain wave signals provided in the above embodiment.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0074] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0075] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for controlling air wave gloves based on brain wave signals, applied to a server of a helmet device, characterized in that: The method comprises: Receive EEG signal output data from the helmet device based on TGAM chip; Analyzing the EEG signal output data to obtain original EEG signal data; Determining EEG signal features according to the original EEG signal data; Inputting the EEG signal features into a preset EEG signal decoding model to determine the user's intended control instructions for the air wave gloves; The air wave gloves are controlled to perform corresponding operation actions according to the intention control instruction.

2. The method according to claim 1, characterized in that Before the step of inputting the EEG signal feature into a preset EEG signal decoding model to determine the user's intended control instruction for the air wave gloves, the method further includes: After determining that the helmet device is turned on by the user and connected to Bluetooth, it is detected that a light is emitted from one side of the Bluetooth connection device, then it is determined that the helmet device has established a communication connection with the air wave glove; Determine the noise value by debugging the equipment; If it is detected that the noise value changes from 0 to 100, it is determined that the user's wearing contact is poor, and the speaker is controlled to issue a reminder message to put it on again; If the noise value displayed changes from full to 0 when the user uses the helmet device, it means that the user is wearing it correctly.

3. The method according to claim 1, characterized in that After the step of receiving the EEG signal output data sent by the helmet device based on the TGAM chip, it also includes: Detecting whether the checksum of the packet data in the EEG signal output data is correct; If correct, the original EEG signal data is determined; If it is incorrect, the corresponding data packet is discarded; After the packet loss rate is determined, if the packet loss rate is lower than a set threshold, the alarm device is controlled to issue an alarm message.

4. The method according to claim 1, characterized in that: After the step of parsing the EEG signal output data to obtain the original EEG signal data, the method further includes: Extracting and screening the raw EEG signal data to determine effective concentration data and relaxation data of the user; If the effective concentration data and the relaxation data control the strength of the air wave gloves.

5. The method according to claim 1, characterized in that The step of controlling the air wave glove to perform corresponding operation actions according to the intention control instruction specifically includes: Determining the to-be-executed components of the air wave glove according to the intention control instruction; Determine the required action type corresponding to the to-be-executed component; After determining the air wave glove control instruction according to the action type and the component to be executed, the air wave glove control instruction is sent to the air wave glove so that the air wave glove controls the pressure and flow of the component to be executed to complete the action type.

6. The method according to claim 1, characterized in that After the step of controlling the air wave gloves to perform corresponding operation actions according to the intention control instruction, the method further includes: Continue to acquire the original EEG signal data, and extract EEG signal features from the original EEG signal data; Determining the user's satisfaction with the action performed by the air wave glove according to the extracted EEG signal features; If the EEG signal feature indicates that the user is not satisfied with the current result of the action being performed, then reacquiring a new EEG signal feature of the user; The intention control instruction is updated according to the new EEG signal characteristics and sent to the air wave glove.

7. The method according to claim 1, characterized in that Before the step of determining the EEG signal features according to the original EEG signal data, the method further includes: Collect raw EEG signal sample data from multiple users and train a general domain EEG signal feature recognition model; When receiving information that a new user uses the helmet device, collecting new raw EEG signal data of the new user; The general domain EEG signal recognition model is used as a pre-trained model, and the pre-trained model is adjusted according to the new original EEG signal data to obtain a user-personalized EEG signal decoding model suitable for the new user.

8. A helmet device, characterized in that: The helmet device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the helmet device to execute the method described in any one of claims 1-7.

9. A computer program product comprising instructions, characterized in that When the computer program product runs on a helmet device, the helmet device is enabled to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the helmet device, the helmet device is caused to execute the method as described in any one of claims 1 to 7.

Images