Game interactive control method, device, equipment and medium based on electroencephalogram signal

Abstract

The present application discloses a method, device, equipment and medium for interactive game control based on electroencephalogram (EEG) signals, which segment EEG signals according to fixed lengths to obtain multiple EEG units; obtain semantic descriptions of brain activities and core semantic descriptions corresponding to the EEG units, and construct a core semantic set based on the core semantic descriptions to determine the feature vector corresponding to each EEG unit; splice each feature vector according to time sequence; obtain an activity path of dynamically changing brain activities based on multiple EEG units and the spliced ​​feature vectors, match them with multiple patterns of a preset standard pattern library, obtain a pattern area that is highly correlated with the activity path, generate a search range, generate a sub-pattern set in the multiple patterns corresponding to the search range, obtain a confidence score for each pattern in the sub-pattern set and the activity path, determine a target pattern in the multiple patterns, output a game control instruction corresponding to the target pattern, and complete interactive game control of the user.

Description

Technical Field

[0001] The present invention belongs to the technical field of brain-computer interface, and in particular, relates to a game interactive control method, device, equipment and medium based on electroencephalogram signals. Background Art

[0002] In the field of gaming, the application of brain-computer interface technology can bring users a new interactive experience, enabling them to control game characters and scenes directly with their thoughts without tedious physical operations, thereby improving the immersion and entertainment of the game.

[0003] Traditional brain-computer interface game control methods usually adopt the idea based on pattern recognition, that is, first extract some typical characteristic patterns from the EEG signal, such as P300 wave, visual evoked potential (VEP), etc., and then map these patterns one by one with predefined game control instructions. For example, some studies use P300 waves to select game menus, and determine the menu items selected by users by detecting the P300 waves induced when users concentrate their attention; some studies use SSVEP (steady-state visual evoked potential) to control the movement direction of game characters, and determine the direction in which users want to move by analyzing the SSVEP signals generated when users look at targets flashing at different frequencies.

[0004] Although these methods have achieved the mapping of EEG signals to game controls to a certain extent, there are still some limitations: first, these methods usually rely on specific experimental paradigms and stimulation methods, such as P300 matrix, SSVEP scintillator, etc., and users need to undergo certain training to adapt to them, and the operation is not natural and flexible enough; second, these methods mainly rely on a single EEG pattern, such as P300 wave or SSVEP, ignoring other cognitive and emotional information contained in EEG signals, and it is difficult to achieve more fine-grained and diversified game interactions; third, these methods are often fixed and static in the extraction and recognition of EEG patterns, lack the ability to adapt to changes in user status, and are difficult to cope with real-time dynamic interactions during the game. Summary of the invention

[0005] The embodiments of the present application provide a game interaction control method, device, equipment and medium based on EEG signals. The method can comprehensively utilize multi-dimensional information in EEG signals, including cognition, emotion, decision-making and other aspects; it should be able to adaptively extract and identify dynamic changes in user status, and realize real-time intent decoding and feedback control; it should be able to flexibly adapt to the characteristics and needs of different users and different game scenarios, and provide personalized interaction optimization.

[0006] In a first aspect, an embodiment of the present application provides a game interaction control method based on EEG signals, comprising:

[0007] Acquire an electroencephalogram (EEG) signal of a user to be controlled, segment the EEG signal according to a preset fixed length, and acquire a plurality of EEG units;

[0008] Parsing each of the EEG units, obtaining a semantic description of brain activity corresponding to each of the EEG units, determining a core semantic description among a plurality of the semantic descriptions of brain activity, and forming a core semantic set according to each of the core semantic descriptions;

[0009] Determine the feature vector corresponding to each of the EEG units in the core semantic set, splice each of the feature vectors according to the time sequence, and obtain the activity path of the dynamic change of brain activity according to the multiple EEG units and the spliced ​​feature vectors;

[0010] According to the matching of the activity path with multiple patterns of a preset standard pattern library, a pattern area highly related to the activity path is obtained, and a search scope is generated according to the pattern area, the core semantic set and the activity path; the search scope includes multiple patterns in the pattern area and adjacent pattern areas;

[0011] Among the multiple patterns corresponding to the search scope, a sub-pattern set is generated according to the active path, and a confidence score of each of the patterns in the sub-pattern set and the active path is obtained;

[0012] A target mode is determined among the multiple modes according to the confidence score, and a game control instruction corresponding to the target mode is output to complete the game interaction control of the user.

[0013] In a second aspect, the present application further provides a game interactive control device, comprising:

[0014] A signal segmentation unit, used to obtain an electroencephalogram signal of a user to be controlled, segment the electroencephalogram signal according to a preset fixed length, and obtain a plurality of electroencephalogram units;

[0015] A core determination unit, configured to parse each of the EEG units, obtain a semantic description of brain activity corresponding to each of the EEG units, determine a core semantic description from a plurality of the semantic descriptions of brain activity, and form a core semantic set according to each of the core semantic descriptions;

[0016] A vector determination unit, used to determine a feature vector corresponding to each of the EEG units according to the core semantic set, splice each of the feature vectors according to a time sequence, and obtain an activity path of dynamic changes in brain activity according to the plurality of EEG units and the spliced ​​feature vectors;

[0017] A pattern matching unit, configured to match the activity path with a plurality of patterns of a preset standard pattern library, obtain a pattern region highly related to the activity path, and generate a search scope according to the pattern region, a core semantic set and the activity path; the search scope includes a plurality of patterns in the pattern region and adjacent pattern regions;

[0018] A set generating unit, configured to generate a sub-pattern set according to the active path from among the multiple patterns corresponding to the search scope, and obtain a confidence score between each of the patterns in the sub-pattern set and the active path;

[0019] A target determination unit is used to determine a target mode among the multiple modes according to the confidence score, output a game control instruction corresponding to the target mode, and complete the game interaction control of the user.

[0020] In a third aspect, the present application further provides a computer device, comprising a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the game interaction control method based on EEG signals as described in the first aspect is implemented.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the game interaction control method based on EEG signals as described in the first aspect.

[0022] Compared with the prior art, this application has at least the following beneficial effects:

[0023] 1. Improved the accuracy of brain-controlled game interaction. In the data acquisition stage, the present invention uses a high-density, full-coverage electrode arrangement scheme, and introduces a variety of noise reduction and enhancement technologies, such as wavelet transform, to ensure the quality and stability of EEG signals. In the preprocessing stage, the system adopts a sliding window segmentation strategy to improve the processing efficiency while ensuring signal integrity. In the feature extraction stage, the system comprehensively analyzes the time domain, frequency domain, and spatial characteristics of the EEG signal to maximize the retention of effective information in the signal. In the semantic analysis stage, the system uses latent semantic analysis (LSA) to extract topics and compress semantics in the semantic description library, realizing effective mapping from EEG signals to semantic information. In the pattern matching stage, the system uses an improved DTW algorithm to overcome the individual differences and non-stationary characteristics of EEG signals and improve the accuracy of pattern recognition. Through the comprehensive application of the above series of technical means, the present invention significantly improves the accuracy of brain-controlled game interaction, and users can accurately control game characters and scenes through their thinking intentions.

[0024] 2. The real-time performance of brain-controlled game interaction is improved. In the semantic feature extraction stage, the present invention constructs a similarity matrix by calculating the similarity between the feature vector of the activity path and the feature vector of each pattern in the extended retrieval range. In the matching optimization stage, the system uses a dynamic programming algorithm to search for the optimal sub-pattern set in the similarity matrix, ensuring the real-time performance of the matching process. In the control mapping stage, the system converts the representative patterns in the optimal sub-pattern set into game control instructions in real time according to the pre-defined rule base, and performs smooth processing and legality checks on the instructions, ensuring the consistency and real-time performance of the instruction output. Through the above technical means, the present invention significantly improves the real-time performance of brain-controlled game interaction, and users can control game characters and scenes naturally, smoothly and in real time through their thinking intentions, and obtain an immersive feeling.

[0025] 3. Improved the adaptability of brain-controlled game interaction. In the semantic analysis stage, the present invention measures the correlation between the time-frequency domain characteristics of the EEG unit and the semantic description through methods such as the chi-square test, identifies the semantic description that is highly correlated with the specific EEG pattern, and realizes the adaptive understanding of the user's cognitive state and intention. In the pattern matching stage, the system uses weighted average and other methods based on the similarity matrix to screen out the sub-pattern set with the highest matching degree, and dynamically adjusts the matching strategy in combination with the semantic analysis results to adapt to changes in the user's state. In the control mapping stage, the system supports users to customize game instructions and operation methods, and adaptively optimizes the mapping rules based on the user's historical behavior data. Through the above adaptive mechanism, the present invention can make brain-controlled game interaction better adapt to the cognitive characteristics and operating habits of different users, and improve the flexibility of interaction and user experience.

[0026] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a method for interactive game control based on EEG signals according to an embodiment of the present application;

[0028] Figure 2 This is a schematic diagram of the structure of a game interaction control device shown in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of a computer device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0031] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0032] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0033] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0034] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0035] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0036] The technical solution of the embodiment of the present application is introduced below.

[0037] With the continuous development of human-computer interaction technology, brain-computer interface (BCI) as a new interaction method is receiving more and more attention and research. Brain-computer interface aims to establish a direct communication channel between the human brain and external devices, allowing users to control devices through EEG signals and realize human-computer interaction of "thinking is control".

[0038] In the field of gaming, the application of brain-computer interface technology can bring users a new interactive experience, allowing them to control game characters and scenes directly with their thoughts, without tedious physical operations, and improving the immersion and entertainment of the game. However, due to the complexity, variability and individual differences of EEG signals, how to accurately, real-time and stably identify user intentions and convert them into game control instructions still faces many technical challenges.

[0039] Traditional brain-computer interface game control methods usually adopt the idea based on pattern recognition, that is, first extract some typical characteristic patterns from the EEG signal, such as P300 wave, visual evoked potential (VEP), etc., and then map these patterns one by one with predefined game control instructions. For example, some studies use P300 waves to select game menus, and determine the menu items selected by users by detecting the P300 waves induced when users concentrate their attention; some studies use SSVEP (steady-state visual evoked potential) to control the movement direction of game characters, and determine the direction in which users want to move by analyzing the SSVEP signals generated when users look at targets flashing at different frequencies.

[0040] Although these methods have achieved the mapping of EEG signals to game controls to a certain extent, there are still some limitations: first, these methods usually rely on specific experimental paradigms and stimulation methods, such as P300 matrix, SSVEP scintillator, etc., and users need to undergo certain training to adapt to them, and the operation is not natural and flexible enough; second, these methods mainly rely on a single EEG pattern, such as P300 wave or SSVEP, ignoring other cognitive and emotional information contained in EEG signals, and it is difficult to achieve more fine-grained and diversified game interactions; third, these methods are often fixed and static in the extraction and recognition of EEG patterns, lack the ability to adapt to changes in user status, and are difficult to cope with real-time dynamic interactions during the game.

[0041] Therefore, in order to fully tap the application potential of brain-computer interface technology in the field of games, it is necessary to develop a more intelligent, natural and efficient EEG signal game interaction method. This method should be able to comprehensively utilize the multi-dimensional information in EEG signals, including cognition, emotion, decision-making and other aspects; it should be able to adaptively extract and identify the dynamic changes of user status, realize real-time intention decoding and feedback control; it should be able to flexibly adapt to the characteristics and needs of different users and different game scenarios, and provide personalized interaction optimization.

[0042] Please refer to Figure 1 , Figure 1 The flowchart of a method for interactive game control based on EEG signals provided in the embodiment of the present application is shown in FIG. The method for interactive game control based on EEG signals in the embodiment of the present application can be applied to computer devices, including but not limited to smart phones, laptops, tablet computers, desktop computers, physical servers, cloud servers and other devices. Figure 1 As shown, the game interaction control method based on EEG signals of this embodiment includes steps S101 to S106, which are described in detail as follows:

[0043] Step S101, obtaining the EEG signal of the user to be controlled, segmenting the EEG signal according to a preset fixed length, and obtaining a plurality of EEG units.

[0044] Specifically, the user's brain activity is captured using an EEG device to obtain raw EEG signal data. Continuous EEG signals are segmented according to a preset fixed time length to form multiple independent EEG units. Segmenting long-term EEG signals is beneficial for subsequent data analysis and processing. Segmented processing can reduce the impact of signal noise and improve the accuracy and stability of signal processing.

[0045] Step S102, parse each EEG unit, obtain the semantic description of brain activity corresponding to each EEG unit, determine the core semantic description among multiple brain activity semantic descriptions, and form a core semantic set according to each core semantic description.

[0046] Specifically, the characteristic signals of brain activity are extracted by parsing each segmented EEG unit. According to the parsing results, the semantic description of brain activity corresponding to each EEG unit is generated. The core semantic descriptions with representative and dominant meanings are extracted from multiple semantic descriptions of brain activities to form a core semantic set. By converting complex EEG data into easy-to-understand semantic descriptions, the user's intention is made clearer. By extracting the core semantics, redundant information can be filtered out, improving the efficiency and accuracy of the model.

[0047] Step S103, determining the feature vector corresponding to each EEG unit in the core semantic set, splicing each feature vector according to the time sequence, and obtaining the activity path of the dynamic change of brain activity according to multiple EEG units and the spliced ​​feature vectors.

[0048] Specifically, the feature vector corresponding to each EEG unit is determined according to the core semantic set. The feature vectors are spliced ​​in chronological order to form a continuous feature vector sequence. The dynamic change path (activity path) of the user's brain activity is constructed based on multiple EEG units and the spliced ​​feature vectors. Through a series of spliced ​​feature vectors, the system can capture and record the changes in the user's brain activity in real time to form a dynamic activity path. The splicing of feature vectors and path generation facilitates data integration and modeling, and provides continuous activity information.

[0049] Step S104, matching the active path with multiple patterns in a preset standard pattern library, obtaining a pattern area highly related to the active path, and generating a search scope based on the pattern area, the core semantic set and the active path; the search scope includes multiple patterns in the pattern area and adjacent pattern areas.

[0050] Specifically, the active path is matched with multiple patterns in the preset standard pattern library to find the most relevant pattern area. According to the matching results, the pattern area and its adjacent pattern areas are determined, and a search range including these areas is generated. By matching with the standard pattern library, the existing data model can be effectively utilized to reduce the training and computing costs. The generated determined search range narrows the search space and improves the search efficiency and accuracy.

[0051] Step S105 : generating a sub-pattern set according to the active path from among the multiple patterns corresponding to the search scope, and obtaining the confidence score of each pattern in the sub-pattern set and the active path.

[0052] Specifically, among the multiple patterns corresponding to the search scope, several sub-pattern sets are generated according to the activity path. The confidence score of each pattern in the sub-pattern set and the activity path is calculated. Through the confidence score, the pattern that best matches the activity path can be screened out to ensure the accuracy of recognition. The generation and scoring of sub-pattern sets enables the system to perform detailed analysis of brain activity and improve the sensitivity of the model.

[0053] Step S106, determining a target mode from multiple modes according to the confidence scores, outputting game control instructions corresponding to the target mode, and completing game interactive control of the user.

[0054] Specifically, according to the confidence score, the target mode that best matches the user's intention is determined from multiple modes. The target mode is converted into the corresponding game control instruction and sent to the game system to realize the game interaction control of the user. By generating control instructions according to the target mode, the user's EEG signal control needs can be fed back and responded to in real time. It is ensured that the game control instructions are highly consistent with the user's real intention, which improves the authenticity and immersion of the interactive experience. The provided method realizes accurate and real-time game interaction control through fine processing and intelligent analysis of EEG signals, significantly improves the user experience, and may be applied to other brain-computer interface (BCI) related interactive systems.

[0055] In some embodiments, the EEG signal is segmented according to a preset fixed length to obtain multiple EEG units, including: in the EEG signal, the segmentation window is moved forward according to a preset step size to obtain original units of the preset window length; the segmentation window is moved backward according to a first preset time length to obtain original units of another preset window length in the EEG signal; the above steps are repeated to obtain multiple original units, each original unit is extended forward and backward according to a second preset time length, and the extended original unit is reconstructed by wavelet to obtain multiple EEG units.

[0056] In the EEG signal, the split window is moved forward according to a preset step length to obtain the original unit of the preset window length. The split window is moved backward according to the first preset time length to obtain another original unit of the preset window length in the EEG signal. Repeat the above steps to obtain multiple original units. Each original unit is extended forward and backward according to the second preset time length. The extended original unit is reconstructed by wavelet to obtain multiple EEG units. Through the flexible setting of the step size and window length based on the segmentation technology, the signal segment is more in line with the needs of practical applications. Wavelet reconstruction can effectively remove noise, enhance signal quality, and improve the accuracy and reliability of subsequent signal processing.

[0057] In some embodiments, each EEG unit is parsed to obtain a semantic description of brain activity corresponding to each EEG unit, including: performing wavelet packet decomposition on each EEG unit to obtain time-frequency domain features corresponding to the EEG unit; inputting the time-frequency domain features into a preset convolutional neural network model to output high-level semantic features corresponding to the time-frequency domain features; inputting the high-level semantic features into a preset bidirectional long short-term memory network to output a semantic description of brain activity.

[0058] Perform wavelet packet decomposition on each EEG unit to extract its time-frequency domain features. Input the time-frequency domain features into the preset convolutional neural network (CNN) model to output high-level semantic features. Input the high-level semantic features into the bidirectional long short-term memory network (Bi-LSTM) to output the semantic description of brain activity. Wavelet packet decomposition combined with CNN can deeply mine the time-frequency domain features and high-level semantic features of the signal. Use Bi-LSTM to capture timing information, so that the semantic description is more consistent with the actual brain activity characteristics.

[0059] In some embodiments, a core semantic description is determined among multiple brain activity semantic descriptions, including: constructing a semantic description library based on each EEG unit and the corresponding brain activity semantic description; performing topic extraction and semantic compression on the semantic description library according to a latent semantic analysis method to generate a low-dimensional semantic topic space; calculating the similarity between each brain activity semantic description and the remaining brain activity semantic descriptions in the semantic topic space; and determining the core semantic description among multiple brain activity semantics based on the similarity.

[0060] A semantic description library is constructed based on each EEG unit and the corresponding semantic description of brain activity. The semantic description library is subjected to topic extraction and semantic compression through latent semantic analysis (LSA) to generate a low-dimensional semantic topic space. In the semantic topic space, the similarity between each semantic description and other semantic descriptions is calculated. The core semantic descriptions in multiple brain activity semantics are determined based on the similarity. The LSA technology can effectively reduce the semantic dimension, extract the main semantic topics, and enhance the simplicity and accuracy of the semantic representation. Through similarity calculation, the most representative core semantic description can be effectively identified to improve recognition accuracy.

[0061] In some embodiments, an activity path of dynamic changes in brain activity is obtained based on multiple EEG units and spliced ​​feature vectors, including: inputting the spliced ​​feature vector into a preset hidden Markov model, and outputting a hidden state sequence corresponding to the feature vector; aligning and fusing the hidden state sequence with multiple EEG units to map each EEG unit to at least one continuous hidden state in the hidden state sequence, thereby generating an activity path.

[0062] The concatenated feature vector is input into the Hidden Markov Model (HMM), and the hidden state sequence corresponding to the feature vector is output. The hidden state sequence is aligned and fused with multiple EEG units, so that each EEG unit is mapped to at least one continuous hidden state in the hidden state sequence to generate an activity path. By using the HMM model, the dynamic changes in the brain activity process can be better tracked and the accuracy of path generation can be improved. Through alignment and fusion, it is ensured that the generated activity path can accurately reflect the actual brain activity dynamics.

[0063] In some embodiments, the activity path is matched with multiple patterns of a preset standard pattern library to obtain a pattern area that is highly correlated with the activity path, including: constructing a pattern vector corresponding to each pattern according to the time characteristics, frequency domain characteristics, and spatial characteristics of each pattern; aligning the activity path with each pattern vector based on a dynamic time warping algorithm; calculating the minimum cumulative distance between the activity path and each aligned pattern vector; determining a target vector from multiple pattern vectors according to the minimum cumulative distance; and the pattern corresponding to the target vector constitutes the pattern area.

[0064] According to the time, frequency and space characteristics of each pattern, a pattern vector is formed. Based on the dynamic time warping (DTW) algorithm, the activity path is aligned with each pattern vector. The minimum cumulative distance between the activity path and each aligned pattern vector is calculated. The target vector is determined based on the minimum cumulative distance, and the pattern corresponding to the vector constitutes the pattern region. The DTW algorithm can effectively solve the alignment problem between different time scales and make the matching process more accurate. The calculation of the minimum cumulative distance can quantify the difference between the path and the pattern, which is convenient for efficient matching.

[0065] In some embodiments, a target pattern is determined among multiple patterns based on confidence scores, including: obtaining first keyword distribution vectors of multiple patterns and second keyword distribution vectors of activity paths; calculating cosine similarity between the first keyword distribution vector and the second keyword distribution vector, using the cosine similarity as a semantic relevance score: calculating a comprehensive score based on the semantic relevance score corresponding to each pattern and the confidence score; and determining a target pattern among multiple patterns based on the comprehensive score.

[0066] Obtain the first keyword distribution vector of multiple patterns and the second keyword distribution vector of the activity path. Calculate the cosine similarity of the two keyword distribution vectors as the semantic relevance score. Calculate the comprehensive score based on the semantic relevance score and the confidence score. Determine the most matching target pattern among multiple patterns based on the comprehensive score. Through keyword distribution and cosine similarity calculation, semantic similarity can be effectively matched to improve matching accuracy. The comprehensive scoring mechanism ensures multi-dimensional considerations, provides a more comprehensive decision-making basis, and ensures the accurate selection of the target pattern.

[0067] To facilitate understanding of the specific implementation of the method provided in this application, a complete example is now used to illustrate the specific application of the method provided in this application:

[0068] 1. Segment the collected EEG data into segments of fixed length to obtain multiple EEG units.

[0069] The EEG device is used to collect EEG signals. It has multiple silver-silver chloride electrodes built in, covering all standard electrode positions specified by the international 10-20 system. After the user wears it correctly, turn on the power switch to start the real-time collection of EEG signals. The electrodes on the inside of the headband will detect the bioelectric activity of brain neurons from the surface of the scalp through ion conduction, generating weak potential changes. These tiny raw EEG signals are transmitted to the bioelectric amplifier circuit at the back of the headband through flexible wires. The amplified analog signal is immediately digitized by a 24-bit ΣΔ analog-to-digital converter with a conversion accuracy of up to 0.5μV / bit, which is sufficient to detect tiny EEG changes. In subsequent data processing, the digital sequence can be downsampled or upsampled according to specific needs to adjust the final output sampling rate. The typical output sampling rate is set between 256-512Hz.

[0070] The digitized EEG data is transmitted to the host terminal via the Bluetooth 5.0 protocol, with a transmission distance of up to 100 meters. To ensure the real-time and reliability of data transmission, this application adopts a seamless automatic switching dual-channel design. That is, the main channel is used to transmit real-time data streams, and the backup channel periodically transmits verification and self-test information. Once the main channel is interrupted or data errors occur, it will automatically switch to the backup channel to continue transmission to avoid data loss or information confusion. At the same time, metadata such as timestamps and sequence numbers are embedded in the data stream to ensure the restoration of the correct time sequence, thereby laying the foundation for subsequent processing.

[0071] After the above process, the present application obtains a high-quality continuous EEG data stream from the user's entire brain area. Next, the data stream needs to be segmented to extract meaningful data segments. The present application uses a sliding window segmentation method to segment the data stream with a fixed window length of 1 second and a step length of 800 milliseconds. This parameter setting can ensure a reasonable balance between continuity and time resolution and match the subsequent steps.

[0072] Specifically, the system first moves the segmentation window forward by 800 milliseconds to extract the corresponding 1-second data; then moves the window back by 200 milliseconds to overlap and cut the next 1-second data segment; and repeats this cycle until the data stream ends. In this way, a large number of fixed-length, overlapping data segment sequences are obtained. Compared with non-overlapping segmentation, overlapping segmentation can maximize the continuity and integrity of cognitive activities, while also avoiding information loss.

[0073] In order to eliminate the impact of the segmentation boundary, this application extends each data segment by 100 milliseconds before and after, and uses wavelet reconstruction to achieve smooth transition, ensuring that the waveform near the segmentation point is complete and continuous. In addition, in order to improve system efficiency, this application implements a CUDA-based parallel segmentation algorithm for GPUs with a large amount of parallel computing capabilities. The algorithm distributes the data to be segmented to different threads, and uses the GPU's stream multiprocessor to perform segmentation tasks in large-scale parallel, thereby shortening the segmentation time to about 1 / 100 of the original time, effectively improving the system's real-time processing capabilities. Through the above segmentation process, this application finally obtained a large number of EEG data segment queues that were 1.2 seconds long and overlapped 800 milliseconds.

[0074] 2. Apply the algorithm to analyze each EEG unit obtained in step 1 to obtain the corresponding semantic description of brain activity.

[0075] First, the EEG unit is subjected to wavelet packet decomposition (WPD) to extract the time-frequency domain features. Wavelet packet decomposition can obtain a fine and complete representation on the time-frequency plane by recursively decomposing and reconstructing the signal. Suppose the time domain signal of the EEG unit is x(t), then its wavelet packet decomposition can be expressed as: x(t)=∑∑d_j^k(t), where j is the decomposition scale, k is the frequency band index, and d_j^k(t) is the wavelet packet coefficient on the kth frequency band of the jth scale. The Daubechies wavelet function family (such as db4) is selected as the basis function, and the optimal decomposition scale and frequency band division scheme are determined according to the sampling frequency and duration of the signal.

[0076] For example, for an EEG unit with a sampling frequency of 1000 Hz and a duration of 1 second, the db4 wavelet function can be selected and decomposed in 6 layers to obtain 2^6=64 frequency bands, each with a bandwidth of approximately 1000 / 64≈15.6 Hz and a time resolution of 1 / 64 second. Such parameter settings can better balance the time-frequency resolution and are suitable for capturing typical rhythms and patterns in EEG signals.

[0077] After obtaining the wavelet packet coefficients, a variety of statistical features are extracted for each frequency band, including mean, standard deviation, skewness, kurtosis, etc., to characterize its energy distribution and waveform characteristics. At the same time, the interactive features between frequency bands, such as correlation coefficient, mutual information, phase synchronization, etc., can also be calculated to reflect the functional connection pattern between different brain regions. In order to improve the robustness and reliability of the features, the wavelet packet coefficients usually need to be preprocessed before extracting the features, such as removing artifacts and smoothing noise. Independent component analysis (ICA) is used here for preprocessing, which can separate independent EEG power components from multi-channel EEG signals and effectively remove the influence of physiological artifacts such as electrooculography and electromyography.

[0078] Next, a convolutional neural network (CNN) model is used to extract high-level semantic features from the time-frequency feature map. CNN can automatically learn the spatial invariance and hierarchical structure in the data through operations such as local receptive field, weight sharing, and pooling. Here, a ResNet-based CNN architecture is designed, including multiple convolutional layers, pooling layers, residual blocks, and fully connected layers, which can efficiently map the input time-frequency feature map into a compact semantic embedding vector. At the same time, the introduction of attention mechanism and skip connection enables the network to adaptively focus on important patterns in different time-frequency regions and establish multi-scale feature representations. The CNN model is pre-trained on a large-scale EEG semantic dataset, and a general semantic feature extractor is learned through supervised classification or unsupervised autoencoding tasks.

[0079] Finally, the semantic features extracted by CNN are input into the Bidirectional Long Short-Term Memory (BiLSTM) network to model the entire EEG unit sequence and obtain the final semantic description. By introducing information flows in both forward and backward directions, BiLSTM can simultaneously utilize contextual information and capture long-distance dependencies in time series. On the basis of BiLSTM, the self-attention mechanism and pointer generation network are introduced, so that the model can dynamically focus on key features at different moments and flexibly generate natural language descriptions based on contextual information. At the same time, a training strategy based on reinforcement learning is adopted to guide the model to generate more accurate, fluent and coherent semantic descriptions by setting a reward function. In practical applications, heuristic search algorithms such as Beam Search are usually used to decode the optimal semantic description sequence from the output probability distribution of BiLSTM.

[0080] Given an EEG unit with a duration of 1 second, this method can generate the following semantic description: 'The subject is playing a brain-controlled game based on SSVEP. There are four directional arrows flashing at 12Hz, 8Hz, 15Hz and 20Hz in the game screen, corresponding to the character's forward, backward, left turn and right turn operations respectively. A significant 12Hz SSVEP response is detected in the current EEG unit, indicating that the subject is looking at the arrow in the forward direction and maintaining a stable attention. After the system recognizes the subject's intention, it will control the game character to continue moving forward until the next control instruction arrives. The whole process reflects that the subject achieves real-time control of the game character's movement direction and speed through visual selection of attention.

[0081] 3. From all the semantic descriptions obtained in step 2, combined with the EEG units segmented in step 1, identify the most representative core semantic set.

[0082] First, for each EEG unit, this application extracts its corresponding semantic description and builds a semantic description library. This semantic description library contains the semantic information of all EEG units and is the basis for subsequent analysis. In order to facilitate analysis and comparison, this application needs to preprocess and normalize the semantic description, such as removing stop words, word form restoration, synonym replacement, etc., and convert the description into a standardized Bag-of-Words representation.

[0083] Next, this application uses Latent Semantic Analysis (LSA) to extract topics and compress semantics from the semantic description library. LSA is an unsupervised learning method based on singular value decomposition (SVD), which can automatically discover hidden topics in a text collection and map words and documents into a low-dimensional semantic space. Through LSA, this application can transform a high-dimensional, sparse bag-of-words representation into a low-dimensional, dense semantic vector representation, thereby better characterizing the similarity and correlation between semantic descriptions.

[0084] Before applying LSA, this application needs to construct a term-document co-occurrence matrix (Term-Document Matrix, TDM), in which each row represents a term, each column represents a semantic description, and the matrix elements represent the frequency or weight of the term in the description. Then, singular value decomposition is performed on TDM to obtain the product of three matrices: TDM = U × Σ × V^T, where U and V are orthogonal matrices, Σ is a diagonal matrix, and the elements on the diagonal are singular values. By selecting the first k largest singular values ​​and their corresponding singular vectors, this application can approximate the original TDM to a low-rank matrix: TDM ≈U_k × Σ_k × V_k^T, where k represents the number of selected topics. This low-rank matrix is ​​the semantic topic space extracted by this application, and each semantic description can be represented as a vector in this space.

[0085] With the semantic topic space, the present application can calculate the similarity between different semantic descriptions and group them using a clustering algorithm. Here, the present application uses the k-means clustering algorithm to divide the semantic description vectors into k clusters, so that the similarity of the vectors within the cluster is maximized and the similarity of the vectors between clusters is minimized. By adjusting the value of k, the present application can obtain semantic categories of different granularities, thereby performing multi-scale exploration and analysis of the semantic space.

[0086] However, relying solely on the content of the semantic description is not enough to accurately identify the core semantics, and this application also needs to combine the segmentation information of the EEG unit. Since the time-frequency domain features of each EEG unit have been extracted in step 2, such as the energy and power spectrum density of the δ, θ, α, β, γ and other frequency bands, this application can directly use these features to perform association analysis with the semantic description.

[0087] Specifically, for each EEG unit, the present application concatenates its time-frequency domain features and the corresponding semantic description vector into a combined feature vector. Then, the present application uses the Chi-Square Test to measure the correlation between each time-frequency domain feature and the semantic description. The Chi-Square Test is a non-parametric test method used to measure the independence between two categorical variables. By calculating the Chi-Square statistic and the p-value, the present application can determine whether there is a significant correlation between a certain time-frequency domain feature and the semantic description.

[0088] For example, if the present application finds that the energy of theta rhythm is significantly positively correlated with a certain semantic category (such as "concentration"), then the present application can infer that the semantic category may correspond to the user's concentration state and have a characteristic theta rhythm pattern in the EEG signal. Conversely, if the present application finds that the energy of alpha rhythm is significantly positively correlated with another semantic category (such as "relaxation"), then the present application can infer that the semantic category may correspond to the user's relaxation state and have a characteristic alpha rhythm pattern in the EEG signal.

[0089] By comprehensively analyzing the clustering results of semantic descriptions and the association patterns of EEG units, this application can extract a set of core semantic categories that are the most representative in semantics and closely related to specific EEG patterns.

[0090] 4. Based on the core semantic set obtained in step 3, assign a corresponding feature vector to each EEG unit obtained in step 1.

[0091] Specifically, for each EEG unit, the present application first uses the KNN (K-Nearest Neighbors) algorithm to determine the core semantic category to which it belongs. KNN is a commonly used supervised learning algorithm that determines the category of a sample by calculating the distance between samples. Here, the present application compares the semantic description vector of each EEG unit with the central vector of all core semantic categories, calculates the Euclidean distance between them, and selects the K core semantic categories with the closest distance as the candidate categories of the EEG unit. Then, the present application adopts a weighted voting method to comprehensively consider the distance and frequency of the candidate categories to determine the core semantic category to which the EEG unit ultimately belongs.

[0092] Assuming that this application has m EEG units and n core semantic categories, the semantic description vector of the ith EEG unit is x_i, and the center vector of the jth core semantic category is c_j, then the Euclidean distance between the ith EEG unit and the jth core semantic category is: d(x_i, c_j) = sqrt(sum((x_i - c_j)^2)). This application selects the K core semantic categories with the closest distance as the candidate categories of the ith EEG unit, denoted as N_i. Next, this application calculates the weight of each candidate category, and the weight calculation formula is: w_j = 1 / d(x_i, c_j), j ∈ N_i. That is, the smaller the distance between the candidate category and the semantic description vector of the EEG unit, the greater its weight. Finally, this application normalizes the weights of the candidate categories and selects the category with the largest weight as the final category of the ith EEG unit:

[0093] y_i = argmax(w_j / sum(w_j)), j ∈ N_i. Where y_i represents the core semantic category to which the i-th EEG unit belongs. After determining the core semantic category to which each EEG unit belongs, this application constructs a semantic feature vector for it. The feature vector consists of two parts: the one-hot encoding of the core semantic category and the semantic topic space vector.

[0094] For the one-hot encoding of core semantic categories, this application first assigns a unique index number (such as 1 to n) to each core semantic category, and then sets the corresponding position of each EEG unit to 1, and the remaining positions to 0. For example, assuming that this application has 5 core semantic categories, and the i-th EEG unit belongs to the third category, its one-hot encoding is [0, 0, 1, 0, 0].

[0095] For the semantic topic space vector, this application directly uses the low-dimensional semantic topic vector representation obtained by the LSA algorithm. The LSA algorithm decomposes the word-document co-occurrence matrix into the product of three matrices through singular value decomposition (SVD), where the elements on the diagonal of the second matrix are singular values, representing the importance of the semantic topic. This application selects the left singular vector and the right singular vector corresponding to the first k largest singular values ​​as the vector representation of words and documents in the k-dimensional semantic topic space. Here, this application regards the semantic description of each EEG unit as a document, and directly uses the document semantic topic vector obtained by LSA as the semantic topic space representation of the EEG unit.

[0096] Finally, this application concatenates the one-hot encoding of the core semantic category and the semantic topic space vector into a complete semantic feature vector. Assuming that the length of the one-hot encoding is n and the length of the semantic topic space vector is k, the length of the semantic feature vector of each EEG unit is n+k.

[0097] For example, suppose this application has 10 EEG units, 5 core semantic categories, and the dimension of the semantic topic space is 20. After the KNN algorithm, this application obtains the core semantic category to which each EEG unit belongs and generates the corresponding one-hot encoding. At the same time, this application uses the LSA algorithm to obtain the vector representation of each EEG unit in the 20-dimensional semantic topic space. Finally, this application concatenates the one-hot encoding and the semantic topic space vector to obtain a 25-dimensional semantic feature vector, as shown below:

[0098] [0, 1, 0, 0, 0, 0.12, -0.08, 0.23, ..., 0.05]. The first 5 elements are one-hot encodings, indicating that the EEG unit belongs to the second core semantic category; the last 20 elements are semantic topic space vectors, indicating the position of the EEG unit in the semantic topic space. In this way, the present application can generate a semantic feature vector for each EEG unit, comprehensively characterizing its semantic attributes in discrete core semantic categories and continuous semantic topic space.

[0099] 5. Link the feature vectors obtained in step 4 in chronological order and combine them with the segmented information in step 1 to construct an activity path that reflects the dynamic changes in brain activity.

[0100] First, according to the time sequence of EEG units, their semantic feature vectors are connected into a long vector sequence. Assuming that there are T EEG units in this application, and the dimension of the semantic feature vector of each EEG unit is D, the connected long vector sequence can be expressed as: X = [x_1, x_2, ..., x_T]. Where x_t is the D-dimensional semantic feature vector of the t-th EEG unit (t=1,2,...,T). This long vector sequence actually depicts the evolution trajectory of the user's semantic representation in the time dimension throughout the game process.

[0101] Next, the Hidden Markov Model (HMM) is used to characterize the temporal dependency and state transition rules of the semantic feature vector sequence. HMM assumes that the observation sequence (such as the semantic feature vector sequence) is generated by a hidden Markov chain, and the observation value at each time step only depends on the hidden state of the current time step. By learning the joint probability distribution of the observation sequence and the hidden state sequence, HMM can characterize the temporal correlation between the observation values ​​and the dynamic rules of state transition.

[0102] When applying HMM, the semantic feature vector sequence X is regarded as the observation sequence, and the user's cognitive state and operation intention transition are regarded as the hidden state sequence. Assume that the hidden state set is S={s_1, s_2, ..., s_N}, where N is the number of hidden states, and each hidden state corresponds to a typical cognitive mode or operation intention category, such as "move left", "move right", "jump", "attack", etc. The parameters of HMM include the initial state probability vector π, the state transition probability matrix A, and the observation probability matrix B.

[0103] Given an observation sequence X and HMM parameters θ=(π, A, B), the forward-backward algorithm is used to calculate the likelihood probability P(X|θ) of X under the HMM, that is, the probability that the observation sequence X is generated by the HMM. The forward-backward algorithm recursively calculates the forward probability α and the backward probability β, and then combines them to obtain the likelihood probability.

[0104] Among them, the forward probability α_t(i) represents the probability of being in state s_i at time step t and observing the first t feature vectors x_1, x_2, ..., x_t; the backward probability β_t(i) represents the probability of observing the last Tt feature vectors x_{t+1}, x_{t+2}, ..., x_T under the condition that the state is s_i at time step t.

[0105] Then, the Viterbi algorithm is used to find the hidden state sequence that is most likely to generate the observation sequence X, that is, the optimal path. The Viterbi algorithm recursively calculates the maximum probability δ_t(i) of being in state s_i at time step t and observing the first t feature vectors, as well as the optimal predecessor state ψ_t(i) of being in state s_i at time step t, through dynamic programming.

[0106] By applying HMM and Viterbi algorithm, the semantic feature vector sequence is transformed into an optimal hidden state sequence, which depicts the dynamic transformation of the user's cognitive state and operation intention throughout the game. Each hidden state corresponds to a typical cognitive mode or operation intention category, such as "move left", "move right", "jump", "attack", etc. The optimal path shows the evolution and transformation rules of these cognitive states and operation intentions in the time dimension, reflecting the user's game strategy and behavior pattern.

[0107] Finally, the hidden state sequence obtained by HMM is aligned and fused with the EEG segmentation information in step 1. Specifically, each EEG segment is mapped to one or more continuous hidden states to obtain a segment-state correspondence table. In this way, each node on the activity path can be associated with a specific EEG segment, giving it clear cognitive and operational meaning.

[0108] For example, suppose that the present application obtains the following segment-state correspondence: Segment 1 (0-2 seconds): corresponding to the hidden state sequence [s_1, s_1], indicating that the user is moving to the left Segment 2 (2-5 seconds): corresponding to the hidden state sequence [s_2, s_2, s_2], indicating that the user is moving to the right and avoiding obstacles Segment 3 (5-7 seconds): corresponding to the hidden state sequence [s_3, s_3], indicating that the user is jumping to reach a high platform... Through this mapping relationship, the present application can correspond each node on the activity path to a specific EEG segment, revealing the dominant cognitive state and operational intention in different time periods.

[0109] Combining the above steps, the application finally obtains an activity path that reflects the dynamic changes of the user's brain activity. This activity path consists of a series of nodes, each of which corresponds to an EEG segment and is associated with one or more hidden states, representing the user's main cognitive state and operational intention during the time period. The transfer between nodes reflects the evolution of cognitive state and operational intention in the time dimension, reflecting the user's game strategy and behavior pattern.

[0110] 6. Using the activity path constructed in step 5, perform pattern matching in the standard EEG pattern library to determine the pattern areas that are highly correlated with it.

[0111] The standard EEG pattern library is a pre-built database that contains a large number of typical EEG patterns, each of which corresponds to a specific cognitive state or operational intention.

[0112] The standard EEG pattern can be represented by a triple (F, T, S): F represents the frequency domain feature, that is, the energy distribution of the pattern in different frequency bands. Generally, this application decomposes the EEG signal into the following typical frequency bands: δ (0.5-4Hz): related to deep sleep and unconscious state; θ (4-8Hz): related to light sleep, meditation, memory encoding; α (8-13Hz): related to relaxation, closing eyes, and inhibitory control; β (13-30Hz): related to concentration, thinking, and decision-making; γ (>30Hz): related to perception, consciousness, and learning. For each frequency band, this application can calculate the proportion of its energy to the total energy and obtain a normalized frequency domain feature vector F.

[0113] T represents the time domain feature, that is, the waveform feature of the pattern in the time dimension. Common time domain features include: mean: reflects the average amplitude level of the signal; variance: reflects the fluctuation range of the signal; skewness: reflects the asymmetry of the signal, positive skewness indicates more positive peaks, negative skewness indicates more negative peaks; kurtosis: reflects the sharpness of the signal, high kurtosis indicates more extreme values ​​and spikes; zero crossing rate: reflects the oscillation frequency of the signal, that is, the number of times the signal crosses the zero point; form factor: reflects the complexity of the signal, that is, the degree of difference between the actual waveform and the ideal sine wave. By calculating these time domain features, the present application can obtain a time domain feature vector T that describes the waveform morphology of the signal.

[0114] S represents the spatial feature, that is, the distribution feature of the pattern in different brain regions or electrodes. According to the acquisition method of EEG signals and the arrangement of electrodes, the present application can divide the brain into different regions, such as: Frontal lobe: related to high-level cognitive functions such as attention, planning, and decision-making; Parietal lobe: related to sensory information processing, motor control, and spatial perception; Temporal lobe: related to hearing, language, emotion, memory, and other functions; Occipital lobe: related to visual information processing, image recognition, and other functions. For each brain region, the present application can calculate the average energy or correlation of the signal on its electrode to obtain a feature vector S that reflects the spatial distribution pattern.

[0115] Combining the features of the above three dimensions, this application can use a high-dimensional vector (F, T, S) to represent a standard EEG pattern. This vector describes the typical features of the pattern in the frequency domain, time domain and space, providing rich information for subsequent pattern matching.

[0116] When performing pattern matching, this application uses the dynamic time warping (DTW) algorithm to align and compare the activity path with the patterns in the standard pattern library. The DTW algorithm can adjust the nonlinear mapping relationship between the two time series in the time dimension and find the optimal matching path, thereby overcoming the deformation and delay effects of EEG signals in time.

[0117] Specifically, for the active path P={p_1, p_2, ..., p_M} and the standard pattern Q={q_1, q_2, ..., q_N}, the present application first extracts the feature vector (F, T, S) of each node or segment, and then calculates the distance matrix D between each pair of nodes. The distance calculation can be based on the Euclidean distance, Mahalanobis distance or cosine similarity of the feature vector, for example:

[0118] D(i,j) = sqrt((F_i - F_j)^2 + (T_i - T_j)^2 + (S_i - S_j)^2). Where F_i, T_i, S_i represent the frequency domain features, time domain features and spatial features of node p_i respectively, and F_j, T_j, S_j represent the corresponding features of node q_j.

[0119] With the distance matrix D, the present application can use the dynamic programming algorithm to calculate the minimum cumulative distance DTW(P,Q) between the active path P and the standard pattern Q: DTW(P,Q) = min_w {sqrt(∑_{(i,j)∈w} D(i,j))}. Where w represents a matching path between P and Q, that is, a series of continuous node pairs (i,j) that satisfy the monotonicity and continuity constraints. In order to find the optimal matching path, the present application can use the following dynamic programming formula:

[0120] γ(i,j) = D(i,j) + min{γ(i-1,j-1), γ(i-1,j), γ(i,j-1)}. Wherein, γ(i,j) represents the cumulative distance when node p_i matches node q_j, the initial condition is γ(1,1)=D(1,1), and the boundary condition is: γ(i,1) = ∑_{k=1}^i D(k,1), i=2,3,...,M. γ(1,j) = ∑_{k=1}^j D(1,k), j=2,3,...,N. By filling the cumulative distance matrix γ from bottom to top, the present application can obtain the minimum cumulative distance DTW(P,Q)=γ(M,N) and the corresponding optimal matching path.

[0121] According to the minimum cumulative distance DTW(P,Q), the application can calculate the similarity between the active path P and the standard pattern Q:

[0122] Sim(P,Q) = exp(-DTW(P,Q)^2 / (2σ^2)). Among them, σ is a parameter that controls the similarity scale and can be set based on experience or cross-validation. Finally, this application matches the activity path P with all the patterns in the standard pattern library one by one, and finds several patterns with the highest similarity as the pattern areas that are highly correlated with the activity path P. These pattern areas represent the typical cognitive state and operation intention of the user during the game.

[0123] 7. Expand the highly correlated pattern area determined in step 6, integrate the semantic information in step 3 and the activity path in step 5, and include other adjacent patterns with higher correlation, thereby expanding the search scope.

[0124] Relying solely on the single optimal matching result in step 6 may have certain limitations. On the one hand, due to the complexity and diversity of EEG signals, the user's actual state may not completely conform to a certain standard pattern, but rather be reflected as a mixture and transition of multiple related patterns. On the other hand, the coverage and granularity of the standard pattern library may also be insufficient, and it is impossible to fully describe the subtle changes and individual differences in the user's state.

[0125] Therefore, in order to obtain a more comprehensive and accurate understanding of the user status, this application needs to expand the highly correlated pattern area determined in step 6, comprehensively utilize the semantic information in step 3 and the activity path in step 5, and incorporate other adjacent patterns with higher correlation, thereby expanding the search scope and improving the flexibility and adaptability of status recognition. Specifically, for each highly correlated pattern area, this application can take the following steps to expand:

[0126] 1. Semantic similarity expansion: Using the core semantic categories and their keyword distribution obtained in step 3, calculate the semantic similarity between the pattern area and other patterns. Semantic similarity can be measured based on indicators such as the co-occurrence frequency of keywords and the cosine distance of word vectors, reflecting the degree of association between different patterns in semantic topics. The present application can select patterns whose semantic similarity exceeds a certain threshold as candidate expansion patterns. For example, suppose that the present application identifies a highly relevant pattern A corresponding to the "focus" state in step 6, and its keyword distribution is: A: {focus: 0.3, thinking: 0.2, decision: 0.15, strategy: 0.1, ...} Then, the present application finds another pattern B in the standard pattern library, and its keyword distribution is: B: {thinking: 0.25, analysis: 0.2, judgment: 0.15, focus: 0.1, ...} By calculating the cosine similarity of the keyword distribution, the present application finds that the semantic similarity between patterns A and B is high, reaching 0.8. Therefore, the present application can include pattern B in the candidate expansion pattern set.

[0127] 2. Temporal adjacency expansion: Using the activity path obtained in step 5, analyze the front and back adjacency relationship of the pattern area on the time axis. If a certain adjacent pattern is close to the current pattern in time and forms a continuous transition or recurring pattern on the activity path, then there may be a close state association between them. The present application can select patterns with a temporal adjacency exceeding a certain threshold as candidate expansion patterns. For example, suppose that the present application identifies the following highly correlated pattern sequence in step 6: [Pattern A (focus), Pattern B (thinking), Pattern C (decision-making), Pattern A (focus), Pattern B (thinking)] Among them, patterns A, B, and C correspond to the states of "focus", "thinking", and "decision-making" respectively. By analyzing the temporal adjacency relationship of the sequence on the activity path, the present application finds that there are frequent mutual conversions between patterns A and B, and the duration is short, while pattern C is relatively independent and the duration is long. Therefore, the present application can infer that there may be a closer state association between patterns A and B, and pattern B is included in the candidate expansion pattern set of pattern A.

[0128] 3. Functional relevance expansion: Use game domain knowledge and user behavior logs to analyze the relevance of the pattern area with other patterns in game functions and user behaviors. If a pattern has a high similarity or complementarity with the current pattern in terms of game tasks, operation types, interaction styles, etc., then there may be a potential state association between them. This application can select patterns with functional relevance exceeding a certain threshold as candidate expansion patterns. For example, suppose that this application identifies a highly relevant pattern C corresponding to the "decision-making" state in step 6, and through analyzing user behavior logs, it is found that this pattern often appears in game level selection, equipment purchase and other scenes. Then, this application finds another pattern D in the standard pattern library, which corresponds to the "evaluation" state and often appears in game task rewards, level upgrades and other scenes. Although patterns C and D are not very similar in semantics, they both involve important decision points in the game and have an important impact on the user's selection behavior. Therefore, this application can include pattern D in the candidate expansion pattern set of pattern C.

[0129] Combining the expansion strategies in the above three dimensions, this application can generate a set of candidate expansion patterns for each highly correlated pattern area, which includes other patterns that are highly correlated with the current pattern in terms of semantics, time, function, etc.

[0130] Next, the application needs to screen and sort the candidate expansion patterns to determine the final expansion result. A commonly used screening method is weighted averaging, that is, assigning a weight to each candidate pattern according to the expansion strategy of different dimensions, and then calculating the weighted average score. For example, the application can define the following weight coefficients: semantic similarity weight: w_s; temporal adjacency weight: w_t; functional relevance weight: w_f.

[0131] Then the weighted average score of candidate pattern i is: score_i = (w_s × sim_s_i + w_t × sim_t_i + w_f × sim_f_i) / (w_s + w_t + w_f). Among them, sim_s_i, sim_t_i, and sim_f_i represent the scores of candidate pattern i in semantic similarity, temporal adjacency, and functional relevance, respectively, and they can be mapped to the interval [0,1] by normalization. According to the weighted average score, the present application can sort the candidate extension patterns, select the top k patterns with the highest scores as the final extension results, and include them in the original high-correlation pattern area to form an expanded retrieval range, which includes other patterns that are closely related to the original pattern in terms of semantics, time, function, etc. This expansion strategy helps to improve the robustness and adaptability of state recognition and alleviate the limitations of single pattern matching.

[0132] 8. Within the search scope expanded in step 7, extract the sub-pattern set that has the highest matching degree with the activity path in step 5.

[0133] First, this application needs to represent each pattern and activity path within the extended search scope as a unified feature vector. For the pattern feature vector, this application comprehensively considers its semantic, time domain, frequency domain, space and other aspects. These features can be extracted and constructed based on the semantic information obtained in step 3 and the typical features of the highly correlated pattern obtained in step 6. For the activity path feature vector, this application further extracts some high-level features based on step 5 to characterize its overall pattern and change trend in time, frequency, space and other dimensions.

[0134] After extracting the pattern feature vector and the activity path feature vector, the present application needs to calculate the similarity between them to measure the matching degree of each pattern and the activity path. Commonly used similarity measurement methods include Euclidean distance, cosine similarity, Pearson correlation coefficient, dynamic time warping (DTW), etc. When calculating the similarity, the present application can assign different weights to different types of features to reflect their importance in the matching process. For example, semantic features and time domain features can be given higher weights, while frequency domain features and spatial features can be given lower weights.

[0135] Let the pattern feature vector be x and the activity path feature vector be y, which are respectively composed of semantic feature x_s, time domain feature x_t, frequency domain feature x_f, spatial feature x_p and semantic feature y_s, time domain feature y_t, frequency domain feature y_f, spatial feature y_p. This application can define a weighted similarity function:

[0136] sim(x,y) = w_s·cos(x_s,y_s) + w_t·r(x_t,y_t) + w_f·cos(x_f,y_f) +w_p·(1-d(x_p,y_p));

[0137] Among them, w_s, w_t, w_f, and w_p are the weights of semantic, time domain, frequency domain, and spatial features respectively, and w_s+w_t+w_f+w_p=1.

[0138] By calculating the weighted similarity between each pattern in the extended retrieval range and the active path, the present application obtains a similarity matrix S, where S(i,j) represents the similarity between the i-th pattern and the active path at the j-th node. Next, the present application needs to search for the optimal set of sub-patterns in the similarity matrix S so that it can maximize the coverage and matching of the overall pattern of the active path. This can be viewed as a two-dimensional interval selection problem, where the goal is to select a sub-matrix in S so that the sum of its elements is maximized. To solve this problem, the present application can use a dynamic programming algorithm. Let dp(i,j) represent the maximum sub-matrix sum with S(i,j) as the lower right corner, then the state transition equation is:

[0139] dp(i,j) = max{dp(i-1,j), dp(i,j-1), dp(i-1,j-1)+S(i,j), S(i,j)}. The boundary conditions are: dp(0,j)=S(0,j), dp(i,0)=S(i,0). By filling the dp matrix, this application can obtain the maximum submatrix sum and the corresponding sub-pattern set. Assume that the maximum submatrix sum is max_sum, and the lower right corner position is (i_max, j_max), then the optimal sub-pattern set can be obtained by backtracking the dp matrix: initialize the sub-pattern set P to an empty set; start from (i_max, j_max), if dp(i,j) is equal to dp(i-1,j), then backtrack upward to (i-1,j); if it is equal to dp(i,j-1), then backtrack to the left to (i,j-1); if it is equal to dp(i-1,j-1)+S(i,j), then add the i-th pattern to P and backtrack to the upper left to (i-1,j-1); if it is equal to S(i,j), then add the i-th pattern to P and terminate the backtracking; repeat step 2 until backtracking to the boundary of the dp matrix. Finally, the present application obtains an optimal sub-pattern set P, which can cover and match the overall pattern of the activity path in multiple dimensions such as time, frequency, space, and semantics to the greatest extent.

[0140] 9. Calculate the matching confidence score between each pattern in the sub-pattern set obtained in step 8 and the active path in step 5, and sort them from high to low according to the confidence combined with the semantic information in step 3.

[0141] First, for each pattern p_i in the sub-pattern set P, this application calculates its matching confidence score conf(p_i,A) with the activity path A. This confidence score comprehensively considers the similarity between the pattern and the activity path in multiple dimensions such as time, frequency, space, and semantics, as well as the significance and representativeness of the pattern itself.

[0142] Specifically, the matching confidence score can be defined as the product of the weighted similarity between the pattern and the activity path and the pattern significance: conf(p_i,A) = sim(p_i,A) × sal(p_i). Among them, sim(p_i,A) represents the weighted similarity between pattern p_i and activity path A, and sal(p_i) represents the significance score of pattern p_i.

[0143] The weighted similarity sim(p_i,A) can be calculated in a similar way to step 8, that is, weighted average of different features of the pattern and activity path: sim(p_i,A) = w_s×sim_s(p_i,A) + w_t×sim_t(p_i,A) +w_f×sim_f(p_i,A)+ w_p×sim_p(p_i,A). Among them, sim_s(p_i,A), sim_t(p_i,A), sim_f(p_i,A), sim_p(p_i,A) represent the similarity between pattern p_i and activity path A in semantic, time domain, frequency domain, space and other dimensions respectively, and w_s, w_t, w_f, w_p are the weight coefficients of the corresponding dimensions.

[0144] The similarity calculation can use a metric function similar to step 8, such as cosine similarity, Pearson correlation coefficient, etc. For example, in the semantic dimension, the present application can compare the cosine similarity of the keyword distribution vectors of the pattern and the activity path:

[0145] sim_s(p_i,A) = cos(v_i, v_A). Where v_i and v_A are the TF-IDF keyword weight vectors of pattern p_i and activity path A respectively. The pattern significance score sal(p_i) reflects the prominence and representativeness of pattern p_i in the entire extended search range. This application can comprehensively consider the energy, duration, frequency distribution, spatial distribution and other characteristics of the pattern to calculate a normalized significance index.

[0146] For example, the present application may define the energy significance of pattern p_i as the ratio of its average energy to the average energy of all patterns in the entire search range: sal_e(p_i) = e_i / (1 / N × ∑e_j). Where e_i is the average energy of pattern p_i, and N is the total number of patterns in the search range. Similarly, the present application may define the duration significance sal_d(p_i), frequency distribution significance sal_f(p_i), spatial distribution significance sal_p(p_i), etc. of pattern p_i.

[0147] Then, the significance indexes of each dimension are weighted averaged to obtain the comprehensive significance score of pattern p_i:

[0148] sal(p_i)=w_e×sal_e(p_i)+ w_d×sal_d(p_i) + w_f×sal_f(p_i) + w_p×sal_p(p_i).

[0149] Among them, w_e, w_d, w_f, and w_p are the weight coefficients of each dimension.

[0150] Through the above calculations, this application obtains the matching confidence score conf(p_i,A) between each pattern in the sub-pattern set P and the active path A.

[0151] Next, the application needs to further optimize and refine the sub-pattern set in combination with the semantic information obtained in step 3. Specifically, the application can use the results of semantic analysis to evaluate the semantic category and keyword distribution of each pattern, and calculate its relevance and consistency with the activity path at the semantic level.

[0152] For example, the present application can compare the cosine similarity of the keyword distribution vector v_i of pattern p_i and the keyword distribution vector v_A of activity path A to obtain a semantic relevance score: rel_s(p_i,A) = cos(v_i, v_A).

[0153] Then, the semantic relevance score and the matching confidence score are weighted and fused to obtain the final comprehensive score of pattern p_i:

[0154] score(p_i,A) = λ × conf(p_i,A) + (1-λ) × rel_s(p_i,A). λ is the weight coefficient of the confidence score and the semantic relevance score, which can be set based on experience or cross-validation.

[0155] Finally, this application sorts the sub-pattern set P according to the comprehensive score score(p_i,A) to obtain a pattern sequence arranged from high to low according to matching confidence and semantic relevance: P_sorted = [p_1, p_2, ..., p_k]. Among them, score(p_1,A) ≥ score(p_2,A) ≥ ... ≥ score(p_k,A). This sorted pattern sequence P_sorted represents a group of patterns that best match and are most relevant to the activity path. They describe the user's current state and intention from different angles and levels, and provide rich semantic information and context clues for subsequent decision-making and interaction.

[0156] 10. Select the most representative pattern from the pattern group with the highest confidence in step 9 and map it to the corresponding game control instruction output.

[0157] First, the application needs to determine the pattern group with the highest confidence. The application can select the top N patterns with the highest comprehensive scores as the pattern group with the highest confidence. Here, taking N=3 as an example, it is expressed as: P_top = [p_1, p_2, p_3]. Among them, p_1, p_2, and p_3 are the patterns with the top three comprehensive scores respectively.

[0158] Next, this application needs to select the most representative pattern from P_top. In order to quantify the representativeness of the pattern, this application comprehensively considers factors such as the comprehensive score, time span, spatial distribution and semantic uniqueness of the pattern, and defines a weighted summation representativeness index: representativeness(p_i) = w_score × score(p_i,A) + w_duration × duration(p_i) + w_distribution × distribution(p_i) + w_uniqueness × uniqueness(p_i).

[0159] Among them, w_score, w_duration, w_distribution, and w_uniqueness are the weight coefficients of each factor, which can be set according to experience or cross-validation; score(p_i,A), duration(p_i), distribution(p_i), and uniqueness(p_i) respectively represent the normalized index values ​​of the comprehensive score, duration, spatial distribution breadth, and semantic uniqueness of pattern p_i.

[0160] By calculating the representativeness index value of each mode in P_top, this application selects the mode with the highest index value as the final control command mode: p_command = argmax(representativeness(p_i)), p_i ∈ P_top. After obtaining the control command mode p_command, this application needs to map it to the corresponding game control instructions. Here, this application adopts a rule-based mapping method, pre-defines a set of mapping rules, and maps patterns of specific semantic categories to specific game instructions.

[0161] Taking the first-person shooter game as an example, the present application may define the following mapping rules: 1. If p_command belongs to the "move" category, its direction semantics is further determined and mapped to the corresponding direction movement instructions, such as "move left" → "translate left", "move forward" → "move forward", etc.; 2. If p_command belongs to the "jump" category, it is mapped to the "jump" instruction; 3. If p_command belongs to the "crouch" category, it is mapped to the "crouch" instruction; 4. If p_command belongs to the "shoot" category, its weapon semantics is further determined and mapped to the corresponding shooting instruction, such as "main weapon" → "move forward". 5. If p_command belongs to the "Switch Weapons" category, it is mapped to the "Switch Weapons" command; 6. If p_command belongs to the "Use Props" category, it is mapped to the "Use Props" command; 7. If p_command belongs to the "Interaction" category, its object semantics is further determined and mapped to the corresponding interaction command, such as "Talk to NPC" → "Start Talking", "Pick Up Items" → "Pick Up", etc.; 8. If p_command belongs to the "Other" category, it is mapped to the corresponding custom command according to the specific game requirements.

[0162] The above rules cover the main operations of first-person shooter games and can be adjusted and expanded according to the actual needs of the game.

[0163] After the mapping is completed, the application obtains a game control instruction cmd_game corresponding to the control command mode p_command. Next, the application needs to perform some post-processing and adjustments on cmd_game to improve the stability and playability of the system.

[0164] First, the present application performs smoothing on the continuously output instruction sequence to reduce the jitter and mutation of the instruction. This can be achieved by introducing an instruction buffer and a filtering algorithm. Specifically, the present application maintains an instruction buffer of fixed length, and each time a new instruction is generated, it is added to the end of the buffer, and the instruction at the head of the buffer is popped out; then, the instruction sequence in the buffer is filtered, such as median filtering, mean filtering, etc., to eliminate outliers and noise.

[0165] Secondly, this application performs legality checks and constraints on instructions to ensure that they comply with game rules and physical limitations. For example, if the current instruction will cause the character to move to the map boundary or obstacles, the instruction needs to be corrected or blocked; if the execution time of the current instruction exceeds the game frame rate limit, the instruction needs to be delayed or discarded.

[0166] Thirdly, the application can also optimize and prompt the command sequence in combination with the game's AI system and difficulty curve. For example, if the user continuously inputs a large number of identical or simple commands, the game difficulty can be appropriately increased to improve the enemy's health or attack power; if the user frequently uses certain specific command combinations, they can be identified as combo actions, automatically triggering or prompting more advanced operations.

[0167] Finally, the post-processed game control command cmd_game is output to the game engine or hardware device to realize real-time game interactive control based on EEG signals. After the game engine receives cmd_game, it controls the game character or scene to make corresponding actions and reactions, such as moving, jumping, shooting, interacting, etc., according to the type and parameters of the command.

[0168] In order to implement the game interaction control method based on EEG signals corresponding to the above method embodiment, to achieve the corresponding functions and technical effects. Figure 2 , Figure 2 The structure block diagram of a game interaction control device 200 provided in an embodiment of the present application is shown. For the convenience of description, only the parts related to the present embodiment are shown. The game interaction control device 200 provided in an embodiment of the present application includes:

[0169] The signal segmentation unit 201 is used to obtain the EEG signal of the user to be controlled, segment the EEG signal according to a preset fixed length, and obtain multiple EEG units;

[0170] A core determination unit 202 is used to parse each EEG unit, obtain a brain activity semantic description corresponding to each EEG unit, determine a core semantic description from multiple brain activity semantic descriptions, and form a core semantic set according to each core semantic description;

[0171] A vector determination unit 203 is used to determine a feature vector corresponding to each EEG unit according to the core semantic set, splice each feature vector according to the time sequence, and obtain an activity path of dynamic changes in brain activity according to multiple EEG units and the spliced ​​feature vectors;

[0172] The pattern matching unit 204 is used to match the activity path with multiple patterns of a preset standard pattern library, obtain a pattern area that is highly related to the activity path, and generate a search scope according to the pattern area, the core semantic set and the activity path; the search scope includes multiple patterns in the pattern area and adjacent pattern areas;

[0173] A set generating unit 205 is used to generate a sub-pattern set according to the active path from among the multiple patterns corresponding to the search scope, and obtain a confidence score of each pattern in the sub-pattern set and the active path;

[0174] The target determination unit 206 is used to determine the target mode among the multiple modes according to the confidence score, output the game control instruction corresponding to the target mode, and complete the game interaction control of the user.

[0175] The above-mentioned game interaction control device 200 can implement the game interaction control method based on EEG signals of the above-mentioned method embodiment. The options in the above-mentioned method embodiment are also applicable to this embodiment and will not be described in detail here. The rest of the contents of the embodiment of the present application can refer to the contents of the above-mentioned method embodiment, and will not be repeated in this embodiment.

[0176] Figure 3 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. Figure 3 As shown, the computer device 3 of this embodiment includes: at least one processor 30 ( Figure 3 Only one is shown in the figure), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 implements the steps of any of the above method embodiments when executing the computer program 32.

[0177] The computer device 3 may be a computing device such as a smart phone, a tablet computer, a desktop computer, a cloud server, etc. The computer device may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will understand that Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0178] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0179] In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 31 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0180] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0181] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in the above-mentioned method embodiments when executing the computer device.

[0182] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.

[0183] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0184] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A game interactive control method based on EEG signals, characterized in that: include: Acquire an electroencephalogram (EEG) signal of a user to be controlled, segment the EEG signal according to a preset fixed length, and acquire a plurality of EEG units; Parsing each of the EEG units, obtaining a semantic description of brain activity corresponding to each of the EEG units, determining a core semantic description among a plurality of the semantic descriptions of brain activity, and forming a core semantic set according to each of the core semantic descriptions; Determine the feature vector corresponding to each EEG unit in the core semantic set, splice each feature vector according to the time sequence, and obtain the activity path of the dynamic change of brain activity according to the multiple EEG units and the spliced ​​feature vectors; including: inputting the spliced ​​feature vector into a preset hidden Markov model, and outputting the hidden state sequence corresponding to the feature vector; aligning and fusing the hidden state sequence with the multiple EEG units to map each EEG unit to at least one continuous hidden state in the hidden state sequence, and generating the activity path; According to matching the activity path with multiple patterns of a preset standard pattern library, a pattern area highly related to the activity path is obtained, including: forming a pattern vector corresponding to each pattern according to the time characteristics, frequency domain characteristics and spatial characteristics of each pattern; aligning the activity path with each pattern vector based on a dynamic time warping algorithm; calculating the minimum cumulative distance between the activity path and each aligned pattern vector; determining a target vector from multiple pattern vectors according to the minimum cumulative distance; the pattern corresponding to the target vector constitutes the pattern area; generating a search scope according to the pattern area, the core semantic set and the activity path; the search scope includes multiple patterns in the pattern area and adjacent pattern areas; Among the multiple patterns corresponding to the search scope, a sub-pattern set is generated according to the active path, and a confidence score of each of the patterns in the sub-pattern set and the active path is obtained; A target mode is determined among the multiple modes according to the confidence score, and a game control instruction corresponding to the target mode is output to complete the game interaction control of the user.

2. The method according to claim 1, characterized in that: The step of segmenting the EEG signal according to a preset fixed length to obtain a plurality of EEG units includes: In the EEG signal, the split window is moved forward according to a preset step length to obtain an original unit of a preset window length; Shifting the segmentation window backward according to the first preset time length, obtaining another original unit of the preset window length in the EEG signal; Repeat the above steps to obtain a plurality of the original units, expand each of the original units forward and backward according to a second preset time length, perform wavelet reconstruction on the expanded original units, and obtain a plurality of the EEG units.

3. The method according to claim 1, characterized in that: The step of parsing each of the EEG units to obtain a semantic description of brain activity corresponding to each of the EEG units includes: Performing wavelet packet decomposition on each of the EEG units to obtain time-frequency domain features corresponding to the EEG unit; Inputting the time-frequency domain features into a preset convolutional neural network model, and outputting high-level semantic features corresponding to the time-frequency domain features; The high-level semantic features are input into a preset bidirectional long short-term memory network, and the semantic description of the brain activity is output.

4. The method according to claim 1, characterized in that Determining a core semantic description from the plurality of brain activity semantic descriptions comprises: Constructing a semantic description library according to each of the EEG units and the corresponding semantic description of brain activities; Performing topic extraction and semantic compression on the semantic description library according to a latent semantic analysis method to generate a low-dimensional semantic topic space; Calculating the similarity between each of the brain activity semantic descriptions and the remaining brain activity semantic descriptions in the semantic topic space; The core semantic description is determined among the plurality of brain activity semantics according to the similarity.

5. The method according to claim 1, characterized in that Determining a target mode from among the multiple modes according to the confidence scores comprises: Acquire a plurality of first keyword distribution vectors of the patterns and a second keyword distribution vector of the activity path; Calculate the cosine similarity between the first keyword distribution vector and the second keyword distribution vector, and use the cosine similarity as the semantic relevance score: A comprehensive score is calculated based on the semantic relevance score and confidence score corresponding to each pattern; A target mode is determined among the plurality of modes according to the comprehensive score.

6. A game interactive control device, characterized in that: include: A signal segmentation unit, used to obtain an electroencephalogram signal of a user to be controlled, segment the electroencephalogram signal according to a preset fixed length, and obtain a plurality of electroencephalogram units; A core determination unit, configured to parse each of the EEG units, obtain a semantic description of brain activity corresponding to each of the EEG units, determine a core semantic description from a plurality of the semantic descriptions of brain activity, and form a core semantic set according to each of the core semantic descriptions; A vector determination unit is used to determine the feature vector corresponding to each of the EEG units according to the core semantic set, splice each of the feature vectors according to the time sequence, and obtain the activity path of the dynamic change of brain activity according to the multiple EEG units and the spliced ​​feature vectors; including: inputting the spliced ​​feature vectors into a preset hidden Markov model, and outputting the hidden state sequence corresponding to the feature vector; aligning and fusing the hidden state sequence with the multiple EEG units to map each of the EEG units to at least one continuous hidden state in the hidden state sequence to generate the activity path; A pattern matching unit, used for matching the activity path with a plurality of patterns of a preset standard pattern library to obtain a pattern region highly correlated with the activity path, including: forming a pattern vector corresponding to each pattern according to the time characteristics, frequency domain characteristics and spatial characteristics of each pattern; aligning the activity path with each pattern vector based on a dynamic time warping algorithm; calculating the minimum cumulative distance between the activity path and each aligned pattern vector; determining a target vector from a plurality of pattern vectors according to the minimum cumulative distance; the pattern corresponding to the target vector forming the pattern region; generating a search scope according to the pattern region, a core semantic set and the activity path; the search scope including a plurality of patterns in the pattern region and adjacent pattern regions; A set generating unit, configured to generate a sub-pattern set according to the active path from among the multiple patterns corresponding to the search scope, and obtain a confidence score between each of the patterns in the sub-pattern set and the active path; A target determination unit is used to determine a target mode among the multiple modes according to the confidence score, output a game control instruction corresponding to the target mode, and complete the game interaction control of the user.

7. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 5.

Images