Electroencephalogram signal identification method, device and equipment for medical service and medium
By applying multi-head attention mechanism and Transformer encoder in the brain-computer interface, the problem of low accuracy in classification recognition of EEG signals in the prior art is solved, and higher recognition accuracy and instruction escape accuracy are achieved.
Patent Information
- Application Number
- CN202411581668.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2025-02-14
AI Technical Summary
Under the data limit, existing brain-computer interface and motor imagination models have problems such as insufficient fitting, low classification recognition accuracy, and difficulty in fusing multi-level EEG signal characteristics, which affect the accuracy of instruction escape results.
The multi-head attention mechanism is used to perform feature encoding and attention calculation on the EEG signal. Through the combination of the first Transformer encoder and the second Transformer encoder, hidden features are output and multi-level attention calculation is performed to finally realize the EEG signal. high accuracy classification recognition.
By comprehensively utilizing shallow and deep features, the classification and identification accuracy of EEG signals is improved, and the accurate escape ability of EEG signal instructions is enhanced.
Smart Images
Figure CN119441954A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical services and artificial intelligence technology, and in particular to an electroencephalogram (EEG) signal recognition method, device, equipment and medium for medical services. Background Art
[0002] Motor imagery is the state of imagination of the human brain about the performance of an action, that is, the subject imagines in his mind that he is doing an action (but he is not actually doing it), and judges the action by recording EEG signals. In order to cope with more complex medical service scenarios, medical research has proven that motor imagery and real motor activities have similar neural mechanisms, and both processes utilize the part of the cerebral cortex related to motor sensation. For special groups of people with limb disabilities or paralysis, research and applications related to motor imagery can directly reflect the intention of human brain thinking activities, providing them with a possibility to manipulate wheelchairs or prostheses for free activities without relying on physical movement.
[0003] At present, due to the limitation of data volume, the existing brain-computer interface and motor imagery models have problems such as insufficient fitting degree (low classification and recognition accuracy) and difficulty in integrating multi-level EEG signal features. These problems affect the accuracy of the results after command escape, and thus make the wheelchair unable to move completely and accurately according to the instructions of the controller. Therefore, how to classify and recognize EEG signals in the process of medical services to improve recognition accuracy has become an urgent problem to be solved. Summary of the invention
[0004] In view of this, the embodiments of the present invention provide an EEG signal recognition method, device, equipment and medium for medical services to solve the problem of low accuracy in classifying and identifying EEG signals during medical services.
[0005] In a first aspect, an embodiment of the present invention provides an EEG signal recognition method for medical services, the EEG signal recognition method comprising: Collecting EEG signals to be identified; Performing feature encoding on the EEG signal to be identified to obtain an embedded coding feature, performing position encoding on the EEG signal to be identified to obtain a first position coding feature, inputting the embedded coding feature and the position coding feature into a first Transformer encoder, and outputting a first hidden feature; Performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified; Performing position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, inputting the second position encoding feature and the first hidden feature into a second Transformer encoder, and outputting the second hidden feature; According to the first attention feature, performing attention calculation on the second hidden feature to obtain a second attention feature of the EEG signal to be identified; The second attention feature is classified and identified to obtain a classification and identification result.
[0006] In a second aspect, an embodiment of the present invention provides an electroencephalogram signal recognition device for medical services, the electroencephalogram signal recognition device comprising: An acquisition module, used for acquiring the EEG signal to be identified; A second encoding module is used to perform feature encoding on the EEG signal to be identified to obtain an embedded coding feature, perform position encoding on the EEG signal to be identified to obtain a first position coding feature, input the embedded coding feature and the position coding feature into a first Transformer encoder, and output a first hidden feature; A first calculation module, used for performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified; A second encoding module is used to perform position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, input the second position encoding feature and the first hidden feature into a second Transformer encoder, and output the second hidden feature; a second calculation module, configured to perform attention calculation on the second hidden feature according to the first attention feature, so as to obtain a second attention feature of the EEG signal to be identified; The recognition module is used to classify and recognize the second attention feature to obtain a classification recognition result.
[0007] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer device is used to execute the EEG signal recognition method described in any one of the first aspects.
[0008] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to perform the EEG signal recognition method described in any one of the first aspects.
[0009] Compared with the prior art, the present invention has the following beneficial effects: Collect the EEG signal to be identified, perform feature encoding on the EEG signal to be identified, obtain an embedded coding feature, perform position encoding on the EEG signal to be identified, obtain a first position coding feature, input the embedded coding feature and the position coding feature into the first Transformer encoder, output the first hidden feature, perform attention calculation on the first hidden feature, obtain the first attention feature of the EEG signal to be identified, perform position encoding on the first hidden feature, obtain the second position coding feature of the first hidden feature, input the second position coding feature and the first hidden feature into the second Transformer encoder, output the second hidden feature, perform attention calculation on the second hidden feature according to the first attention feature, obtain the second attention feature of the EEG signal to be identified, classify and identify the second attention feature, and obtain a classification recognition result. In this application, a multi-head attention mechanism is used to perform attention calculation on the EEG signal to be identified, and the result of the first layer of multi-head attention is used as the input of the second layer of multi-head attention, and attention calculation is performed again according to the second hidden feature, which can comprehensively utilize shallow and deep features, thereby improving the accuracy of classification and recognition of the second attention feature. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0011] Figure 1 This is a schematic diagram of an application environment of an electroencephalogram signal recognition method for medical services provided by an embodiment of the present invention; Figure 2 It is a flowchart of a method for recognizing electroencephalogram signals for medical services provided by one embodiment of the present invention; Figure 3 is a structural block diagram of an electroencephalogram signal recognition device for medical services provided by an embodiment of the present invention; Figure 4 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0013] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.
[0014] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0015] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0016] As used in the present specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.
[0017] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0018] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0019] The embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0020] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0021] It should be understood that the order of execution of the steps in the following embodiments does not imply a precedence of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0022] In order to illustrate the technical solution of the present invention, specific embodiments are provided below for illustration.
[0023] An embodiment of the present invention provides an electroencephalogram signal recognition method for medical services, which can be applied in Figure 1 In the application environment, the client communicates with the server. The client includes but is not limited to PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, personal digital assistants (PDAs) and other computer devices. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0024] See also Figure 2 , is a flow chart of a method for recognizing electroencephalogram signals for medical services provided by an embodiment of the present invention. The method for recognizing electroencephalogram signals for medical services can be applied to Figure 1 The server in Figure 2 As shown, the EEG signal recognition method for medical services may include the following steps.
[0025] S201: Collecting the EEG signal to be identified.
[0026] In step S201, an acquisition device is used to acquire brain wave signals to be identified, wherein the brain wave signals to be identified are brain wave signals of a special group of people with limb disabilities or paralysis.
[0027] In this embodiment, a conventional EEG monitoring tool can be used to collect the EEG signal to be identified. The EEG monitoring tool is composed of an EEG cap, electrodes and conductive paste. It should be noted that the EEG signal collected by the EEG monitoring tool is a corresponding analog signal, and the analog signal is converted into a digital signal so as to further process the EEG signal. For example, the analog signal is converted into a digital signal through an analog-to-digital converter to obtain the EEG signal to be identified.
[0028] S202: feature encoding is performed on the EEG signal to be recognized to obtain an embedded coding feature, position encoding is performed on the EEG signal to be recognized to obtain a first position coding feature, the embedded coding feature and the position coding feature are input into a first Transformer encoder, and a first hidden feature is output.
[0029] In step S202, the feature encoder is used to perform feature encoding on the EEG signal to be identified to obtain an embedded coding feature, and the position encoder is used to perform position encoding on the EEG signal to be identified to obtain a first position coding feature, and the embedded coding feature and the position coding feature are input into the first Transformer encoder to output a first hidden feature. The embedded coding feature includes the spatial information and time information of the EEG signal to be identified, and the first position coding feature represents the position information of the EEG signal to be identified.
[0030] In this embodiment, a feature encoder is used to perform feature encoding on the EEG signal to be identified, wherein the feature encoder includes a multi-layer convolutional layer network, such as the feature encoder in the Wav2vec2.0 model. The feature encoder in the Wav2vec2.0 model is composed of a 7-layer convolutional network. GroupNorm is used in the first convolution layer, that is, each channel of the entire EEG signal to be identified is normalized, and LayerNorm is used in the last convolution output layer, that is, the channels in each batch of data are normalized. A position encoder is used to perform position encoding on the EEG signal to be identified to obtain a first position encoding feature, wherein the position encoder can be an encoder composed of a one-dimensional convolutional layer network. The embedded encoding feature and the position encoding feature are input into the first Transformer encoder, and the first hidden feature is output.
[0031] Optionally, feature encoding is performed on the EEG signal to be identified to obtain embedded coding features, including: Obtain a feature encoder, the feature encoder includes a first convolutional network layer, a second convolutional network layer, and a third convolutional network layer; Use the first layer of convolutional network to perform channel normalization on the EEG signal to be recognized to obtain normalized features; Use the second layer of convolutional network to convolve the normalized features to obtain convolutional features; The convolutional features are normalized using the third layer of convolutional network to obtain embedded coding features.
[0032] In this embodiment, the feature encoder includes a first-layer convolutional network, a second-layer convolutional network and a third-layer convolutional network. The first-layer convolutional network is connected to the second-layer convolutional network, the second-layer convolutional network is connected to the third-layer convolutional network, the output of the first-layer convolutional network is used as the input of the second-layer convolutional network, and the output of the second-layer convolutional network is used as the input of the third-layer convolutional network.
[0033] The first convolutional network includes a GroupNorm layer. The first convolutional network is used to normalize each channel in the EEG signal to be recognized to obtain normalized features. The second convolutional network is used to convolve the normalized features to obtain convolutional features. The third convolutional network includes a LayerNorm layer. The third convolutional network is used to normalize the convolutional features to obtain embedded coding features.
[0034] S203: Perform attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified.
[0035] In step S203, based on the attention mechanism, the first hidden feature is subjected to attention calculation to obtain the first attention feature of the EEG signal to be identified, that is, different weight values are assigned to the features corresponding to different signal sequences in the first hidden feature.
[0036] In this embodiment, when performing attention calculation on the first hidden feature, self-attention calculation or other attention calculation can be performed. When performing self-attention calculation, the first hidden feature is multiplied by the same parameter matrix to obtain a query matrix, an index matrix and a value matrix, wherein the query matrix, the index matrix and the value matrix are equal. The query matrix and the index matrix are multiplied to obtain a correlation matrix. The correlation matrix is used as a weight value matrix, and the weight value matrix is multiplied by the value matrix to obtain the first attention feature.
[0037] Optionally, performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified includes: According to the first hidden feature, a first query matrix, a first index matrix and a first value matrix corresponding to the first hidden feature are calculated; According to the query matrix and the index matrix, the first attention weight value is calculated; According to the attention weight value and the value matrix, the first attention feature is calculated.
[0038] In this embodiment, the first hidden feature is multiplied by the first parameter matrix to obtain a first query matrix, the first hidden feature is multiplied by the second parameter matrix to obtain a first index matrix, and the first hidden feature is multiplied by the third parameter matrix to obtain a first value matrix. The first parameter matrix, the second parameter matrix, and the third parameter matrix are parameter matrices obtained in advance. The first parameter matrix, the second parameter matrix, and the third parameter matrix can be the same parameter matrix or different parameter matrices.
[0039] The first query matrix is multiplied by the first index matrix. After the multiplication, in order to prevent the multiplication result from being too large, the first attention weight value is calculated by dividing it by the dimension value of the first query matrix and the first index matrix. The first attention weight value is multiplied by the first value matrix to calculate the first attention feature.
[0040] Optionally, performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified further includes: According to the first hidden feature, N groups of first sub-query matrices, first sub-index matrices and first sub-value matrices corresponding to the first hidden feature are calculated, where N is an integer greater than 1; According to each group of first sub-query matrix and first sub-index matrix, the first sub-attention weight value of each group is calculated; According to each group of first sub-attention weight values and the first sub-value matrix of the corresponding group, N groups of first sub-attention features are calculated; Concatenate N groups of first sub-attention features to obtain the first attention feature.
[0041] In this embodiment, a multi-head attention calculation is used to obtain a first attention feature, and according to the first hidden feature, N groups of first sub-query matrices, first sub-index matrices and first sub-value matrices corresponding to the first hidden feature are calculated, where N is an integer greater than 1. Among them, the first hidden feature is multiplied by the first sub-parameter matrix to obtain a first sub-query matrix, the first hidden feature is multiplied by the second sub-parameter matrix to obtain a first sub-index matrix, and the first hidden feature is multiplied by the third sub-parameter matrix to obtain a first sub-value matrix. Among them, the first sub-parameter matrix, the second sub-parameter matrix and the third sub-parameter matrix are parameter matrices obtained in advance. The first sub-parameter matrix, the second sub-parameter matrix and the third sub-parameter matrix can be the same parameter matrix or different parameter matrices. According to N groups of first sub-parameter matrices, second sub-parameter matrices and third sub-parameter matrices, N groups of first sub-query matrices, first sub-index matrices and first sub-value matrices are calculated.
[0042] Multiply each group of first sub-query matrix with the first sub-index matrix. In order to prevent the multiplication result from being too large, divide it by the dimension value of the first sub-query matrix and the first sub-index matrix to calculate N groups of first sub-attention features. Multiply each group of first sub-attention weight values with the first sub-value matrix of the corresponding group to calculate N groups of first sub-attention features.
[0043] N groups of first sub-attention features are concatenated and transformed through the current projection to obtain the first attention feature.
[0044] S204: Perform position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, input the second position encoding feature and the first hidden feature into a second Transformer encoder, and output the second hidden feature.
[0045] In step S204, the first hidden feature is position-encoded to obtain a second position-encoded feature of the first hidden feature, wherein the position of the first hidden feature is enhanced using the position encoder, the second position-encoded feature and the first hidden feature are input into a second Transformer encoder, and the second hidden feature is output.
[0046] In this embodiment, the first hidden feature is used as an embedded feature and input into a second Transformer encoder together with the second position coding feature, the second hidden feature is output, and feature extraction is performed again on the first hidden feature to improve the accuracy of feature extraction.
[0047] S205: Perform attention calculation on the second hidden feature according to the first attention feature to obtain a second attention feature of the EEG signal to be identified.
[0048] In step S205, based on the attention mechanism, the second hidden feature is subjected to attention calculation to obtain the second attention feature of the EEG signal to be identified, that is, different weight values are assigned to the features corresponding to different signal sequences in the second hidden feature.
[0049] In this embodiment, when performing attention calculation on the second hidden feature, self-attention calculation or other attention calculation can be performed. When performing self-attention calculation, the second hidden feature is multiplied by the same parameter matrix to obtain a query matrix, an index matrix and a value matrix, wherein the query matrix, the index matrix and the value matrix are equal. The query matrix and the index matrix are multiplied to obtain a correlation matrix. The correlation matrix is used as a weight value matrix, and the weight value matrix is multiplied by the value matrix to obtain the second attention feature.
[0050] Optionally, performing attention calculation on the second hidden feature according to the first attention feature to obtain a second attention feature of the EEG signal to be identified includes: According to the first attention feature, a second value matrix and a second index matrix corresponding to the first attention feature are calculated, and according to the second hidden feature, a second index matrix is calculated; Calculating a second attention weight value according to the second query matrix and the second index matrix; The second attention feature is calculated based on the second attention weight value and the second value matrix.
[0051] In this embodiment, the first attention feature is multiplied by the fourth parameter matrix to obtain a second query matrix, the first attention feature is multiplied by the fifth parameter matrix to obtain a second index matrix, and the second hidden feature is multiplied by the sixth parameter matrix to obtain a second query matrix, wherein the fourth parameter matrix, the fifth parameter matrix and the sixth parameter matrix are pre-obtained parameter matrices. The fourth parameter matrix, the fifth parameter matrix and the sixth parameter matrix can be the same parameter matrix or different parameter matrices.
[0052] The second query matrix is multiplied by the second index matrix. After the multiplication, in order to prevent the multiplication result from being too large, the second attention weight value is calculated by dividing it by the dimension value of the second query matrix and the second index matrix. The second attention weight value is multiplied by the second value matrix to calculate the second attention feature.
[0053] Optionally, performing attention calculation on the second hidden feature according to the first attention feature to obtain a second attention feature of the EEG signal to be identified further includes: According to the first attention feature, M second sub-value matrices and M second sub-index matrices are calculated, and according to the second hidden feature, M second sub-query matrices are calculated. The M second sub-query matrices, the M second sub-index matrices and the M second sub-value matrices form M groups of second sub-index matrices, second sub-index matrices and second sub-value matrices; According to each group of second sub-query matrix and second sub-index matrix, each group of second sub-attention weight values is calculated; According to each group of second sub-attention weight values and the second sub-value matrix of the corresponding group, M groups of second sub-attention features are calculated; Concatenate M groups of first sub-attention features to obtain the second attention feature.
[0054] In this embodiment, multi-head attention is used to calculate the second attention feature, and according to the first attention feature, M groups of second sub-index matrices and first sub-value matrices corresponding to the first attention feature are calculated, where M is an integer greater than 1. Among them, the first attention feature is multiplied by the fourth sub-parameter matrix to obtain the second sub-value matrix, the first attention feature is multiplied by the fifth sub-parameter matrix to obtain the second sub-index matrix, and the second hidden feature is multiplied by the sixth sub-parameter matrix to obtain the second sub-query matrix. Among them, the fourth sub-parameter matrix, the fifth sub-parameter matrix and the sixth sub-parameter matrix are parameter matrices obtained in advance. The fourth sub-parameter matrix, the fifth sub-parameter matrix and the sixth sub-parameter matrix can be the same parameter matrix or different parameter matrices. According to the M groups of the fourth sub-parameter matrix, the fifth sub-parameter matrix and the sixth sub-parameter matrix, M groups of the second sub-query matrix, the second sub-index matrix and the second sub-value matrix are calculated.
[0055] Multiply each group of second sub-query matrix with the second sub-index matrix. In order to prevent the multiplication result from being too large, divide it by the dimension value of the second sub-query matrix and the second sub-index matrix to calculate M groups of second sub-attention features. Multiply each group of second sub-attention weight values with the second sub-value matrix of the corresponding group to calculate M groups of second sub-attention features.
[0056] The M groups of second sub-attention features are concatenated and transformed through the current projection to obtain the second attention feature.
[0057] S206: Classify and identify the second attention feature to obtain a classification and identification result.
[0058] In step S206, a classifier is used to classify and identify the second attention feature to obtain a classification and identification result.
[0059] In this embodiment, a classifier is used to classify and identify the second attention feature to obtain a classification recognition result. The classification recognition result is the category of the action required by the EEG signal to be identified, such as the category of moving forward or backward. The classifier can be a pre-trained classifier.
[0060] Optionally, after classifying and identifying the second attention feature and obtaining the classification and identification result, the method further includes: Generate control instructions based on classification and recognition results; Control the movement of the equipment according to the control instructions.
[0061] In this embodiment, a control instruction is generated based on the classification and recognition result, wherein the control instruction is an instruction for controlling the movement of a device, such as an instruction for controlling a wheelchair. According to the control instruction, the movement of the device is controlled, such as controlling the wheelchair to move forward.
[0062] Collect the EEG signal to be identified, perform feature encoding on the EEG signal to be identified, obtain an embedded coding feature, perform position encoding on the EEG signal to be identified, obtain a first position coding feature, input the embedded coding feature and the position coding feature into the first Transformer encoder, output the first hidden feature, perform attention calculation on the first hidden feature, obtain the first attention feature of the EEG signal to be identified, perform position encoding on the first hidden feature, obtain the second position coding feature of the first hidden feature, input the second position coding feature and the first hidden feature into the second Transformer encoder, output the second hidden feature, perform attention calculation on the second hidden feature according to the first attention feature, obtain the second attention feature of the EEG signal to be identified, classify and identify the second attention feature, and obtain a classification recognition result. In this application, a multi-head attention mechanism is used to perform attention calculation on the EEG signal to be identified, and the result of the first layer of multi-head attention is used as the input of the second layer of multi-head attention, and attention calculation is performed again according to the second hidden feature, which can comprehensively utilize shallow and deep features, thereby improving the accuracy of classification and recognition of the second attention feature.
[0063] See also Figure 3 , Figure 3 1 is a block diagram of a device for recognizing electroencephalogram signals for medical services provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 3 The EEG signal recognition device 30 includes a collection module 31 , a second encoding module 32 , a first calculation module 33 , a second encoding module 34 , a second calculation module 35 , and a recognition module 36 .
[0064] The acquisition module 31 is used to acquire the EEG signal to be identified.
[0065] The second encoding module 32 is used to perform feature encoding on the EEG signal to be identified to obtain an embedded coding feature, perform position encoding on the EEG signal to be identified to obtain a first position coding feature, input the embedded coding feature and the position coding feature into the first Transformer encoder, and output a first hidden feature.
[0066] The first calculation module 33 is used to perform attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified.
[0067] The second encoding module 34 is used to perform position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, input the second position encoding feature and the first hidden feature into a second Transformer encoder, and output the second hidden feature.
[0068] The second calculation module 35 is used to perform attention calculation on the second hidden feature according to the first attention feature to obtain the second attention feature of the EEG signal to be identified.
[0069] The recognition module 36 is used to classify and recognize the second attention feature to obtain a classification recognition result.
[0070] Optionally, the first calculation module 33 includes: The first calculation unit is used to calculate, according to the first hidden feature, a first query matrix, a first index matrix, and a first value matrix corresponding to the first hidden feature.
[0071] The second calculation unit is used to calculate a first attention weight value according to the query matrix and the index matrix.
[0072] The third calculation unit is used to calculate the first attention feature according to the first attention weight value and the value matrix.
[0073] Optionally, the first calculation module 33 further includes: The fourth calculation unit is used to calculate, according to the first hidden feature, N groups of first sub-query matrices, first sub-index matrices and first sub-value matrices corresponding to the first hidden feature, where N is an integer greater than 1.
[0074] The fifth calculation unit is used to calculate each group of first sub-attention weight values according to each group of first sub-query matrix and the first sub-index matrix.
[0075] The sixth calculation unit is used to calculate N groups of first sub-attention features according to each group of first sub-attention weight values and the first sub-value matrix of the corresponding group.
[0076] The first concatenation unit is used to concatenate N groups of first sub-attention features to obtain a first attention feature.
[0077] Optionally, the second calculation module 35 includes: The seventh calculation unit is used to calculate the second value matrix and the second index matrix corresponding to the first attention feature according to the first attention feature, and calculate the second query matrix according to the second hidden feature.
[0078] The eighth calculation unit is used to calculate the second attention weight value according to the second query matrix and the second index matrix.
[0079] The ninth calculation unit is used to calculate the second attention feature according to the second attention weight value and the second value matrix.
[0080] Optionally, the second calculation module 35 further includes: The tenth calculation unit is used to calculate M second sub-value matrices and M second sub-index matrices according to the first attention feature, and calculate M second sub-query matrices according to the second hidden feature. The M second sub-query matrices, M second sub-index matrices and M second sub-value matrices constitute M groups of second sub-index matrices, second sub-index matrices and second sub-value matrices.
[0081] The eleventh calculation unit is used to calculate each group of second sub-attention weight values according to each group of second sub-query matrix and the second sub-index matrix.
[0082] The twelfth calculation unit is used to calculate M groups of second sub-attention features according to each group of second sub-attention weight values and the second sub-value matrix of the corresponding group.
[0083] The second splicing unit is used to splice M groups of first sub-attention features to obtain a second attention feature.
[0084] Optionally, the second encoding module 32 includes: The acquisition unit is used to acquire a feature encoder, wherein the feature encoder includes a first convolutional network layer, a second convolutional network layer, and a third convolutional network layer.
[0085] The first normalization unit is used to perform channel normalization processing on the EEG signal to be recognized using the first layer of convolutional network to obtain normalized features.
[0086] The convolution unit is used to perform convolution processing on the normalized features using the second layer of convolutional network to obtain convolution features.
[0087] The second normalization unit is used to normalize the convolutional features using the third convolutional network to obtain embedded coding features.
[0088] Optionally, the above-mentioned EEG signal recognition 30 further includes: The generation module is used to generate control instructions according to the classification recognition results.
[0089] The control module is used to control the movement of the device according to the control instructions.
[0090] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0091] Figure 4 FIG. 1 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned embodiments of the method for recognizing an electroencephalogram signal are implemented.
[0092] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 4 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.
[0093] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0094] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.
[0095] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0096] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.
[0097] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0099] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A method for recognizing electroencephalogram signals for medical services, characterized in that: The EEG signal recognition method comprises: Collecting EEG signals to be identified; Performing feature encoding on the EEG signal to be identified to obtain an embedded coding feature, performing position encoding on the EEG signal to be identified to obtain a first position coding feature, inputting the embedded coding feature and the position coding feature into a first Transformer encoder, and outputting a first hidden feature; Performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified; Performing position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, inputting the second position encoding feature and the first hidden feature into a second Transformer encoder, and outputting the second hidden feature; According to the first attention feature, performing attention calculation on the second hidden feature to obtain a second attention feature of the EEG signal to be identified; The second attention feature is classified and identified to obtain a classification and identification result.
2. The method for recognizing an electroencephalogram signal according to claim 1, wherein: The performing attention calculation on the first hidden feature to obtain the first attention feature of the EEG signal to be identified includes: According to the first hidden feature, a first query matrix, a first index matrix and a first value matrix corresponding to the first hidden feature are calculated; Calculate a first attention weight value according to the query matrix and the index matrix; A first attention feature is calculated based on the first attention weight value and the value matrix.
3. The method for recognizing an electroencephalogram signal according to claim 1, wherein: The step of performing attention calculation on the first hidden feature to obtain the first attention feature of the EEG signal to be identified further includes: According to the first hidden feature, N groups of first sub-query matrices, first sub-index matrices and first sub-value matrices corresponding to the first hidden feature are calculated, where N is an integer greater than 1; According to each group of first sub-query matrix and first sub-index matrix, the first sub-attention weight value of each group is calculated; According to each group of first sub-attention weight values and the first sub-value matrix of the corresponding group, N groups of first sub-attention features are calculated; Concatenate N groups of first sub-attention features to obtain the first attention feature.
4. The method for recognizing an electroencephalogram signal according to claim 1, wherein: The step of performing attention calculation on the second hidden feature according to the first attention feature to obtain the second attention feature of the EEG signal to be identified includes: According to the first attention feature, a second value matrix and a second index matrix corresponding to the first attention feature are calculated, and according to the second hidden feature, a second query matrix is calculated; Calculating a second attention weight value according to the second query matrix and the second index matrix; A second attention feature is calculated based on the second attention weight value and the second value matrix.
5. The method for recognizing an electroencephalogram signal according to claim 1, wherein: The method of performing attention calculation on the second hidden feature according to the first attention feature to obtain the second attention feature of the EEG signal to be identified further includes: According to the first attention feature, M second sub-value matrices and M second sub-index matrices are calculated, and according to the second hidden feature, M second sub-query matrices are calculated. The M second sub-query matrices, the M second sub-index matrices and the M second sub-value matrices form M groups of second sub-index matrices, second sub-index matrices and second sub-value matrices; According to each group of second sub-query matrix and second sub-index matrix, each group of second sub-attention weight values is calculated; According to each group of second sub-attention weight values and the second sub-value matrix of the corresponding group, M groups of second sub-attention features are calculated; Concatenate M groups of first sub-attention features to obtain the second attention feature.
6. The method for recognizing an electroencephalogram signal according to claim 1, wherein: The step of encoding the electroencephalogram signal to be identified to obtain an embedded coding feature includes: Acquire a feature encoder, wherein the feature encoder includes a first convolutional network layer, a second convolutional network layer, and a third convolutional network layer; Using the first convolutional network layer to perform channel normalization processing on the EEG signal to be identified to obtain normalized features; Using the second convolutional network layer to perform convolution processing on the normalized features to obtain convolution features; The convolutional features are normalized using the third-layer convolutional network to obtain embedded coding features.
7. The method for recognizing electroencephalogram signals according to claim 1, wherein: After classifying and identifying the second attention feature and obtaining the classification and identification result, the method further includes: generating a control instruction according to the classification and recognition result; According to the control instructions, the movement of the device is controlled.
8. An electroencephalogram signal recognition device for medical services, characterized in that: The electroencephalogram signal recognition device comprises: An acquisition module, used for acquiring the EEG signal to be identified; A second encoding module is used to perform feature encoding on the EEG signal to be identified to obtain an embedded coding feature, perform position encoding on the EEG signal to be identified to obtain a first position coding feature, input the embedded coding feature and the position coding feature into a first Transformer encoder, and output a first hidden feature; A first calculation module, used for performing attention calculation on the first hidden feature to obtain a first attention feature of the EEG signal to be identified; A second encoding module is used to perform position encoding on the first hidden feature to obtain a second position encoding feature of the first hidden feature, input the second position encoding feature and the first hidden feature into a second Transformer encoder, and output the second hidden feature; a second calculation module, configured to perform attention calculation on the second hidden feature according to the first attention feature, so as to obtain a second attention feature of the EEG signal to be identified; The recognition module is used to classify and recognize the second attention feature to obtain a classification recognition result.
9. A computer device, characterized in that: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the computer device is used to execute the EEG signal recognition method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for recognizing electroencephalogram signals as claimed in any one of claims 1 to 7 is implemented.