Classification processing method, device and equipment of electroencephalogram signal and medium

Abstract

The application relates to the technical field of digital medical treatment, in particular to a classification processing method and device of electroencephalogram signals, equipment and a medium. The method acquires electroencephalogram signals with time sequence attributes, divides the electroencephalogram signals according to a preset time length, determines N sub-signals after the division, extracts power spectrum densities of each sub-signal on a preset frequency band, converts the electroencephalogram signals into electroencephalogram time-frequency diagrams according to the preset frequency band, N power spectrum densities and the time sequence relationship of the N sub-signals, extracts noise from the electroencephalogram time-frequency diagrams to obtain characteristic information, constructs a harmonic function based on preset harmonic parameters and at least two types of activation functions, fuses all the characteristic information by using the harmonic function, and determines that the electroencephalogram signals belong to a noise category corresponding to a preset noise distribution when the fusion result meets a preset condition, so that the determination of whether the electroencephalogram signals are noise is realized, the noise category is analyzed, and the accuracy in subsequent medical digital scenarios such as emotion recognition and instruction transfer is improved.

Description

TECHNICAL FIELD

[0001] The present application is suitable for the field of digital medical technology, and particularly relates to a classification processing method and device for electroencephalogram signals, equipment and a medium. BACKGROUND

[0002] With the development of electronics and digitization, applying digitization to the medical industry to promote medical digitization and intelligence has become the main concern. At present, electroencephalogram signals are spontaneous electrical signals generated by human brain consciousness activities, which are digitized information obtained by medical devices and other equipment in a non-invasive manner on the human body. Research on the identification and classification of electroencephalogram signals can be applied to emotion recognition, instruction transfer and other purposes. The original electroencephalogram signals contain various noises (artifacts) generated by external factors (such as cable frequency, subject head movement, click position offset, etc.) and internal factors (such as blinking, eye rotation, facial muscle movement, heartbeat, breathing, etc.). The collected data needs to be preprocessed in a complex manner, which often relies on experience and is time-consuming and labor-intensive, thereby causing the noise classification of electroencephalogram signals to be complex, and the noise classification of electroencephalogram signals cannot be accurately and quickly realized, which leads to the inability to reduce the noise in the electroencephalogram signals, which may affect the use of the electroencephalogram signals in the corresponding scene. For example, in the emotion recognition scene, noise-free electroencephalogram signals are needed to improve the accuracy of emotion recognition.

[0003] Therefore, how to classify the noise of the electroencephalogram signals and improve the accuracy in the use scene has become a problem to be solved. SUMMARY

[0004] In view of this, the present application provides a classification processing method, device, equipment and medium for electroencephalogram signals to solve the problem of how to classify the noise of the electroencephalogram signals and improve the accuracy in the use scene.

[0005] In a first aspect, the present application provides a classification processing method for electroencephalogram signals, which comprises:

[0006] obtaining an electroencephalogram signal with time series attributes, dividing the electroencephalogram signal according to a preset time length, determining N sub-signals after the division, N being an integer greater than zero;

[0007] extracting the power spectral density of each sub-signal on a preset frequency band to obtain N power spectral densities, converting the electroencephalogram signal into an electroencephalogram time-frequency diagram according to the preset frequency band, the N power spectral densities and the time sequence relationship of the N sub-signals, extracting noise features from the electroencephalogram time-frequency diagram to obtain feature information representing the noise;

[0008] construct a harmonic function based on preset harmonic parameters and at least two types of activation functions, fuse all feature information using the harmonic function to obtain a fusion result, the preset harmonic parameters are constructed according to a preset noise distribution;

[0009] When it is detected that the fusion result meets a preset condition, it is determined that the electroencephalogram signal belongs to a noise category corresponding to the preset noise distribution.

[0010] In an embodiment, the electroencephalogram signal is divided according to a preset time length, and the N divided sub-signals are determined as follows:

[0011] A sliding window of a preset time length is used to slide along the time sequence of the electroencephalogram signal according to a preset sliding step;

[0012] The signal in the sliding window at each sliding stop is determined as a divided sub-signal, and N divided sub-signals are obtained.

[0013] In an embodiment, after obtaining the electroencephalogram signal with a time sequence attribute, the following steps are further included:

[0014] The electroencephalogram signal is selected using a preset channel to obtain a selected electroencephalogram signal;

[0015] The selected electroencephalogram signal is down-sampled to obtain a down-sampled electroencephalogram signal;

[0016] The electroencephalogram signal is divided according to a preset time length, including:

[0017] The down-sampled electroencephalogram signal is divided according to a preset time length.

[0018] In an embodiment, noise feature extraction is performed on the electroencephalogram time-frequency graph to obtain feature information representing noise, including:

[0019] The electroencephalogram time-frequency graph is input into a time-frequency feature extraction model to output feature information representing noise;

[0020] The time-frequency feature extraction model includes a convolution layer, a bidirectional long short-term memory layer, and a long short-term memory layer, and the electroencephalogram time-frequency graph is input into the time-frequency feature extraction model to output feature information representing noise, including:

[0021] The electroencephalogram time-frequency graph is input into the convolution layer to output a convolution result;

[0022] The convolution result is input into the bidirectional long short-term memory layer to output a preliminary feature extraction result, and the preliminary feature extraction result is input into the long short-term memory layer to output feature information representing noise.

[0023] In an embodiment, constructing the harmonic function based on a preset harmonic parameter and at least two types of activation functions comprises:

[0024] obtaining a noise variance of a preset noise distribution;

[0025] calculating a standard deviation of the noise variance, normalizing the standard deviation, and determining a result of the normalization as the harmonic parameter;

[0026] obtaining a Softplus function and a ReLU function, taking the harmonic parameter as a weight value of the Softplus function, and taking a difference between the harmonic parameter and 1 as a weight value of the ReLU function;

[0027] taking a weighted sum result of the Softplus function and the ReLU function as the harmonic function.

[0028] In a second aspect, an embodiment of the present application provides a classification processing device for electroencephalogram signals, the classification processing device comprising:

[0029] a signal preprocessing module configured to obtain electroencephalogram signals with time sequence attributes, split the electroencephalogram signals according to a preset time length, and determine N split sub-signals, N being an integer greater than zero;

[0030] a feature extraction module configured to extract power spectral densities of each sub-signal respectively in a preset frequency band, obtain N power spectral densities, convert the electroencephalogram signals into an electroencephalogram time-frequency graph according to the preset frequency band, the N power spectral densities, and a time sequence relationship of the N sub-signals, extract noise features from the electroencephalogram time-frequency graph, and obtain feature information representing the noise;

[0031] a feature fusion module configured to construct a harmonic function based on a preset harmonic parameter and at least two types of activation functions, fuse all feature information using the harmonic function, and obtain a fusion result, the preset harmonic parameter being constructed according to a preset noise distribution;

[0032] a classification processing module configured to determine that the electroencephalogram signals belong to a noise category corresponding to the preset noise distribution when it is detected that the fusion result meets a preset condition.

[0033] In an embodiment, the signal preprocessing module comprises:

[0034] a sliding unit configured to use a sliding window with a preset time length and slide along a time sequence of the electroencephalogram signals according to a preset sliding step;

[0035] a splitting unit configured to determine a signal in the sliding window at each sliding stop as a split sub-signal, and obtain N split sub-signals.

[0036] In an embodiment, the classification processing apparatus further comprises:

[0037] A first preprocessing unit configured to, after obtaining the electroencephalogram signal with time series attribute, select the electroencephalogram signal using a preset channel to obtain a selected electroencephalogram signal.

[0038] A second preprocessing unit configured to down-sample the selected electroencephalogram signal to obtain a down-sampled electroencephalogram signal.

[0039] The signal preprocessing module comprises:

[0040] A third preprocessing unit configured to split the down-sampled electroencephalogram signal according to a preset time length.

[0041] In an embodiment, the feature extraction module comprises:

[0042] A power spectral density acquisition unit configured to perform discrete Fourier transform on each sub-signal in a preset frequency band to obtain a power spectral density of the corresponding sub-signal.

[0043] In an embodiment, the feature extraction module comprises:

[0044] A feature extraction unit configured to input the electroencephalogram time-frequency graph into a time-frequency feature extraction model to output feature information representing noise.

[0045] The time-frequency feature extraction model comprises a convolution layer, a bidirectional long short-term memory layer, and a long short-term memory layer, and the feature extraction unit comprises:

[0046] A convolution subunit configured to input the electroencephalogram time-frequency graph into the convolution layer to output a convolution result.

[0047] A feature extraction subunit configured to input the convolution result into the bidirectional long short-term memory layer to output a preliminary feature extraction result, and input the preliminary feature extraction result into the long short-term memory layer to output the feature information representing noise.

[0048] In an embodiment, the feature fusion module comprises:

[0049] A variance acquisition unit configured to acquire a noise variance of a preset noise distribution.

[0050] A harmonic parameter determination unit configured to calculate a standard deviation of the noise variance, normalize the standard deviation, and determine a normalized result as a harmonic parameter.

[0051] The weight value determination unit is configured to obtain a Softplus function and a ReLU function, take the harmonic parameter as a weight value of the Softplus function, and take a difference between the harmonic parameter and 1 as a weight value of the ReLU function.

[0052] The harmonic function determination unit is configured to take a weighted sum result of the Softplus function and the ReLU function as a harmonic function.

[0053] In a third aspect, an embodiment of the present application provides a computer device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the classification processing method according to the first aspect when executing the computer program.

[0054] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the classification processing method according to the first aspect.

[0055] Compared with the prior art, the embodiment of the present application has the beneficial effects that: the embodiment of the present application obtains the electroencephalogram signal with time sequence attributes, splits the electroencephalogram signal according to a preset time length, determines N sub-signals after splitting, extracts power spectral density on each sub-signal in a preset frequency band, obtains N power spectral densities, converts the electroencephalogram signal into an electroencephalogram time-frequency graph according to the preset frequency band, the N power spectral densities, and a time sequence relationship of the N sub-signals, extracts noise features from the electroencephalogram time-frequency graph to obtain feature information representing the noise, constructs a harmonic function based on a preset harmonic parameter and at least two types of activation functions, fuses all the feature information using the harmonic function to obtain a fusion result, and determines that the electroencephalogram signal belongs to a noise category corresponding to a preset noise distribution when detecting that the fusion result meets a preset condition, thereby realizing the determination of whether the electroencephalogram signal is noise and analyzing the noise category, which is helpful to improve the accuracy in subsequent scenes such as emotion recognition and instruction transfer. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0057] Figure 1 is an application environment schematic diagram of the classification processing method for the electroencephalogram signal provided by the first embodiment of the present application;

[0058] Figure 2is a flowchart of a classification processing method of an electroencephalogram signal provided by Embodiment Two of the present application;

[0059] Figure 3 is a flowchart of a classification processing method of an electroencephalogram signal provided by Embodiment Three of the present application;

[0060] Figure 4 is a structural diagram of a classification processing device of an electroencephalogram signal provided by Embodiment Four of the present application;

[0061] Figure 5 is a structural diagram of a computer device provided by Embodiment Five of the present application. DETAILED DESCRIPTION

[0062] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the application with unnecessary detail.

[0063] It should be understood that the term "includes" when used in the specification and the appended claims herein, specifies the presence of stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0064] It should also be understood that the term "and / or" when used in the specification and the appended claims herein, means and encompasses any and all possible combinations of one or more of the associated listed items and can be used interchangeably with the term "or".

[0065] As used in the description of the application and the appended claims herein, the term "if' can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting [the described condition or event]" or "in response to detecting [the described condition or event]", depending on the context.

[0066] In addition, in the description of the application and the appended claims herein, the terms "first", "second", "third", etc. are used only to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0067] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in additional embodiments," and so on, in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily referring to some, but not all, embodiments. The terms "including," "comprising," "having," and variations thereof are meant to encompass the items listed thereafter, but do not exclude other items from also being present. Unless otherwise indicated, the terms "including," "comprising," and "having" are meant to encompass the items listed thereafter, but do not exclude other items from also being present.

[0068] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by a digital computer, perceiving an environment, acquiring knowledge and using the knowledge to obtain optimal results.

[0069] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0070] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0071] In order to illustrate the technical solutions of the present application, the following will be explained by specific embodiments.

[0072] The brain electrical signal classification processing method provided by the first embodiment of the present application can be applied in application environments such as Figure 1 The client and the server communicate to form a cloud-edge joint system. The client includes but is not limited to a palmtop computer, a desktop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a server computer device, a personal digital assistant (PDA), etc. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0073] Referring to Figure 2is a flowchart of a brain electrical signal classification processing method provided by Embodiment Two of the present application, and the brain electrical signal classification processing method is applied to a server in Figure 1 The server corresponds to a computer device connected to a corresponding database to obtain corresponding brain electrical data in the database. The computer device can also be connected to a corresponding client, and the client can also collect corresponding brain electrical data for the server.

[0074] In medical applications, the brain electrical signal can be collected by an electroencephalogram collection instrument, specifically brain electrical values collected in a continuous time period. The server can be connected to the electroencephalogram collection instrument to directly obtain the brain electrical signal, or the brain electrical signal can be stored in the database for the server to obtain. The classification of the brain electrical signal can be divided into noise and non-noise classes to correct the brain electrical signal and improve the accuracy of the brain electrical signal.

[0075] As shown in Figure 2 The brain electrical signal classification processing method can include the following steps:

[0076] Step S201: Obtain a brain electrical signal with a time sequence attribute, divide the brain electrical signal according to a preset time length, and determine N sub-signals after the division.

[0077] In the present application, N is an integer greater than zero. The brain electrical signal can refer to the brain wave fluctuation of a user measured by an electroencephalogram instrument, specifically an electroencephalogram or a digital signal corresponding to the electroencephalogram.

[0078] The time sequence attribute can refer to a sequence with a time attribute, i.e., the brain electrical signal is a time sequence signal.

[0079] According to the preset time length, the sub-signals are divided from the starting position of the brain electrical signal with the preset time length as the step size. For example, the brain electrical signal is a signal with a time length of 1s, the preset time length is 20ms, and the brain electrical signal is uniformly divided into 50 sub-signals with a time length of 20ms according to the step size of 20ms.

[0080] In addition, a window with a preset time length can be used to divide the sub-signals by sliding the window on the brain electrical signal according to the preset step size. For example, the brain electrical signal is a signal with a time length of 1s, the window is composed of a preset time length of 20ms, and the window is slid on the brain electrical signal according to a step size of 10ms, which can divide the brain electrical signal into 100 sub-signals with a time length of 20ms.

[0081] Optionally, dividing the brain electrical signal according to the preset time length to determine N sub-signals after the division includes:

[0082] A preset length sliding window is used to slide along the time sequence of the electroencephalogram signal according to a preset sliding step;

[0083] The signal in the sliding window at each sliding stop is determined as a segmented sub-signal, and N segmented sub-signals are obtained.

[0084] The sub-signals are segmented in the form of a sliding window, which can make the signal segmentation more random and the information of the sub-signals more abundant, providing rich samples for subsequent training.

[0085] Optionally, after obtaining the electroencephalogram signal with time sequence attributes, the method further includes:

[0086] The electroencephalogram signal is selected using a preset channel to obtain a selected electroencephalogram signal.

[0087] The selected electroencephalogram signal is down-sampled to obtain a down-sampled electroencephalogram signal.

[0088] The electroencephalogram signal is segmented according to a preset length, including:

[0089] The down-sampled electroencephalogram signal is segmented according to a preset length.

[0090] The preset channel can be a channel formed by a filter, etc. The electroencephalogram signal is selected using this channel to obtain a selected electroencephalogram signal, filtering certain noise signals and glitch signals.

[0091] Down-sampling can process multi-rate signals. Since the electroencephalogram signal is collected by multiple different electrode pieces to collect the changes of the brain electricity at different positions of the brain, the generated signal can exist in stacking. Using down-sampling to process the electroencephalogram signal can effectively reduce the occurrence of signal stacking and improve the accuracy of subsequent processing.

[0092] In step S202, the power spectral density of each sub-signal is extracted on a preset frequency band to obtain N power spectral densities. According to the preset frequency band, the time sequence relationship of the N power spectral densities and the N sub-signals, the electroencephalogram signal is converted into an electroencephalogram time-frequency diagram. The noise feature of the electroencephalogram time-frequency diagram is extracted to obtain feature information representing noise.

[0093] In the present application, the corresponding power spectral density on different frequency bands is not the same for brain waves, and different frequency bands in brain wave research can reflect different states of the human body at the time of sampling. For example, for the following five frequency bands, namely alpha wave (alpha wave), beta wave (beta wave), gamma wave (gamma wave), delta wave (delta wave), and theta wave (theta wave), specifically, delta wave (0.5-4Hz), theta wave (4-8Hz), alpha wave (8-12Hz), beta wave (12-30Hz), and gamma wave (30-47Hz), delta wave is related to deep relaxation and restorative sleep, theta wave is related to spiritual trance or hypnotic state, alpha wave can represent the "frequency bridge" between conscious thinking (beta) and subconscious (theta), beta wave is the most common high-frequency wave in wakefulness, and is used to represent conscious states such as cognitive reasoning, calculation, reading, communication, and thinking, and gamma wave appears more frequently in other states.

[0094] Therefore, according to the requirement, the corresponding frequency band is set, and the power spectral density of each sub-signal is extracted on the frequency band. According to the preset frequency band, the time sequence relationship of the N power spectral densities and the N sub-signals, the electroencephalogram is converted into an electroencephalogram time-frequency diagram, noise feature extraction is performed on the electroencephalogram time-frequency diagram, and feature information representing noise is obtained.

[0095] When performing noise feature extraction on the electroencephalogram time-frequency diagram, a time-frequency feature extraction model can be used, that is, the electroencephalogram time-frequency diagram is input into the time-frequency feature extraction model, and the feature information of the noise is output. The time-frequency feature extraction model can refer to a model for extracting time-frequency features, which is constructed based on principles such as short-time Fourier transform, wavelet transform, and Hilbert transform. The model is a trained model, for example, the architecture of the model can use a model such as a long short-term memory model and a bidirectional long short-term memory model that can effectively process time series data.

[0096] Optionally, extracting the power spectral density of each sub-signal on the preset frequency band comprises:

[0097] Discrete Fourier transform is performed on each sub-signal on the preset frequency band to obtain a transform result as the power spectral density of the corresponding sub-signal.

[0098] Among them, the discrete Fourier transform is used to perform transform processing on the sub-signal, and the result of the transform processing is the power spectral density of the sub-signal.

[0099] Optionally, the noise feature extraction on the electroencephalogram time-frequency diagram comprises:

[0100] The electroencephalogram time-frequency diagram is input into the time-frequency feature extraction model, and the feature information representing the noise is output.

[0101] The time-frequency feature extraction model comprises a convolution layer, a bidirectional long short-term memory layer and a long short-term memory layer, the electroencephalogram time-frequency diagram is input into the time-frequency feature extraction model, and feature information representing noise is output, which comprises:

[0102] The electroencephalogram time-frequency diagram is input into the convolution layer, and a convolution result is output.

[0103] The convolution result is input into the bidirectional long short-term memory layer, and a preliminary feature extraction result is output, and the preliminary feature extraction result is input into the long short-term memory layer, and the feature information representing noise is output.

[0104] The convolution layer can be three one-dimensional convolution layers, and the long short-term memory layer and the bidirectional long short-term memory layer are each one layer. The time-frequency feature extraction model can be jointly trained with a subsequent fully connected layer to reduce the difficulty of labeling and improve the accuracy of the time-frequency feature extraction model in the application scenario.

[0105] In step S203, a harmonic function is constructed based on a preset harmonic parameter and at least two types of activation functions, and all feature information is fused using the harmonic function to obtain a fusion result.

[0106] In this application, the preset noise distribution is a noise distribution set according to the judgment requirement, that is, for environmental noise, sound noise in the environment corresponds to one noise distribution, temperature noise in the environment corresponds to one noise distribution, for users, pulse beats and electromyographic responses also produce noise to electroencephalogram signals, so biological noise such as pulse noise and electromyographic noise can also be set, and for collection instruments, mode conversion noise and transmission noise of the instruments also correspond to corresponding noise distributions.

[0107] In order to determine whether the electroencephalogram signal is one or more of the above-mentioned noises, the preset noise distribution can be set to contain multiple noise distributions, and all noise distributions need to be traversed in the above-mentioned step S203.

[0108] In order to represent the correlation between the electroencephalogram signal and the preset noise distribution, a harmonic parameter is constructed using the preset noise distribution, a harmonic function is constructed using the harmonic parameter, and the feature vector of the electroencephalogram signal is fused (i.e., fused using a fully connected layer, and the activation function of the fully connected layer is the harmonic function) using the harmonic function, and the fusion result is used to represent the correlation between the electroencephalogram signal and the preset noise distribution.

[0109] The harmonic function is composed of at least two activation functions, and the harmonic parameter is used to harmonize the weights of the at least two activation functions. The activation function can include Sigmoid function, Tanh function, ReLU function, Leaky ReLU function, RReLU function, Softsign function, Softplus function, Softmax, GELU, threshold function, step function, piecewise linear function, etc.

[0110] Optionally, constructing the harmonic function based on the preset harmonic parameter and the at least two types of activation functions comprises:

[0111] Obtaining a noise variance of the preset noise distribution;

[0112] Calculating a standard deviation of the noise variance, normalizing the standard deviation, and determining a normalized result as the harmonic parameter;

[0113] Obtaining a Softplus function and a ReLU function, taking the harmonic parameter as a weight value of the Softplus function, and taking a difference between the harmonic parameter and 1 as a weight value of the ReLU function;

[0114] Taking a weighted sum result of the Softplus function and the ReLU function as the harmonic function.

[0115] The ReLU function formula is as follows:

[0116] ReLU(x) = max(0, x)

[0117] The Softplus function formula is as follows:

[0118] Softplus(x) = log(1 + e x )

[0119] It can be seen that the Softplus function can be regarded as a smoothed version of the ReLU function.

[0120] Taking the harmonic parameter as η, the corresponding harmonic function formula is as follows:

[0121] y = (1 - η)max(0, x) + ηlog(1 + e x )

[0122] It can be seen that when the harmonic parameter approaches 0, the response approaches the ReLU function, and when the harmonic parameter approaches 1, the response approaches the Softplus function.

[0123] The harmonic parameter is a normalized result of a standard deviation of a noise variance of the preset noise distribution, that is, when the standard deviation approaches 0, the response approaches the ReLU function, and when the standard deviation approaches 1, the response approaches the Softplus function. Compared with the existing electroencephalogram multi-modal feature fusion, the present application can adaptively adjust the activation function of the full connection layer to adapt to different noise distributions and has higher explainability.

[0124] Step S204, when it is detected that the fusion result meets the preset condition, determining that the electroencephalogram signal belongs to a noise category corresponding to the preset noise distribution.

[0125] In the present application, the preset noise distribution corresponds to a corresponding noise category, i.e. the above-mentioned environmental noise category, biological noise category, and instrument noise category. Of course, the mapping relationship can include at least two cases of including and excluding, i.e. according to the fusion result, it can be determined whether the preset noise distribution exists in the electroencephalogram signal.

[0126] For example, if the noise category of the preset noise distribution is environmental noise, and if the classification of the fusion result is "yes", it can be considered that the electroencephalogram signal exists environmental noise, and if the classification of the fusion result is "no", it can be considered that the electroencephalogram signal does not exist environmental noise.

[0127] If the fusion result meets the preset condition, it is determined that the electroencephalogram signal belongs to the noise signal mapped by the noise category corresponding to the preset noise distribution; if the fusion result does not meet the preset condition, it is determined that the electroencephalogram signal does not belong to the noise signal mapped by the noise category corresponding to the preset noise distribution.

[0128] The preset condition can be obtained by training the training set, and the condition for classifying the fusion result. The fusion result meeting the preset condition can mean that the fusion result is similar to the existing fusion result meeting the preset condition. The fusion result not meeting the preset condition can mean that the fusion result is similar to the existing fusion result not meeting the preset condition.

[0129] In the present application, the electroencephalogram signal with time sequence attribute is obtained, the electroencephalogram signal is divided according to the preset time length, N sub-signals after division are determined, the power spectral density of each sub-signal is extracted on the preset frequency band, N power spectral densities are obtained, the electroencephalogram signal is converted into electroencephalogram time-frequency diagram according to the preset frequency band, N power spectral densities and the time sequence relationship of N sub-signals, the noise feature extraction is performed on the electroencephalogram time-frequency diagram, the feature information representing the noise is obtained, the harmonic function is constructed based on the preset harmonic parameter and at least two types of activation function, the fusion of all feature information is performed using the harmonic function, the fusion result is obtained, when it is detected that the fusion result meets the preset condition, it is determined that the electroencephalogram signal belongs to the noise category corresponding to the preset noise distribution, the determination of whether the electroencephalogram signal is noise is realized, and the noise category is analyzed, which is helpful to improve the accuracy in subsequent scenes such as emotion recognition and instruction transfer.

[0130] Referring to Figure 3 , a flowchart of a classification processing method of an electroencephalogram signal provided by Embodiment Three of the present application is shown as Figure 3 , the classification processing method comprises:

[0131] In step S301, an electroencephalogram signal with time sequence attribute is obtained, the electroencephalogram signal is divided according to a preset time length, and N sub-signals after division are determined.

[0132] Step S302, power spectrum densities of each sub-signal are extracted on a preset frequency band, N power spectrum densities are obtained, the electroencephalogram is converted into an electroencephalogram time-frequency diagram according to the preset frequency band, the N power spectrum densities and the time sequence relationship of the N sub-signals, noise feature extraction is performed on the electroencephalogram time-frequency diagram, and feature information representing noise is obtained.

[0133] Step S303, a harmonic function is constructed based on a preset harmonic parameter and at least two types of activation functions, and all feature information is fused using the harmonic function to obtain a fusion result.

[0134] Among them, steps S301 to S303 are the same as the contents of steps S201 to S203 described above, and the specific description can be referred to the description of steps S201 to S203, which will not be repeated here.

[0135] Step S304, if the fusion result does not satisfy the preset condition, it is determined that the electroencephalogram signal does not belong to the noise signal of the preset noise distribution corresponding to the noise category mapping.

[0136] In this application, if the electroencephalogram signal does not belong to the noise signal of the preset noise distribution corresponding to the noise category mapping, it may be a noise signal of the noise category corresponding to other noise distribution, so step S305 is executed to realize the discrimination of other noise distribution.

[0137] If the fusion result satisfies the preset condition, it is determined that the electroencephalogram signal belongs to the noise signal of the preset noise distribution corresponding to the noise category mapping, at this time, step S305 can also be executed to discriminate with other noise distribution, so that the relationship between the electroencephalogram signal and other noise distribution can be known.

[0138] Step S305, other noise distribution is obtained, new harmonic parameter is constructed with other noise distribution, new harmonic function is constructed with new harmonic parameter and two activation functions, all features are fused according to new harmonic function to obtain new fusion result.

[0139] In this application, when the electroencephalogram signal is judged again using other noise distribution, the harmonic function needs to be reconstructed to realize adaptive harmonic and adaptive judgment, which reduces the interference of human participation. The specific process is the same as the content of step S303 described above, which will not be repeated here.

[0140] Step S306, when it is detected that the fusion result satisfies the preset condition, it is determined that the electroencephalogram signal belongs to the noise category corresponding to other noise distribution.

[0141] Among them, step S306 is the same as the content of step S204 described above, and the specific description can be referred to the description of step S204, which will not be repeated here.

[0142] The embodiment of the application obtains an electroencephalogram signal with a time sequence attribute, splits the electroencephalogram signal according to a preset time length, determines N sub-signals after splitting, extracts power spectral density of each sub-signal on a preset frequency band respectively, obtains N power spectral densities, converts the electroencephalogram signal into an electroencephalogram time-frequency diagram according to the preset frequency band, the N power spectral densities and a time sequence relationship of the N sub-signals, extracts noise features of the electroencephalogram time-frequency diagram, obtains feature information representing noise, constructs a harmonic function based on a preset harmonic parameter and at least two types of activation functions, fuses all feature information using the harmonic function, obtains a fusion result, and if the fusion result does not satisfy a preset condition, determines that the electroencephalogram signal is not a noise signal corresponding to a noise category mapping of a preset noise distribution, obtains another noise distribution, constructs a new harmonic parameter using the other noise distribution, constructs a new harmonic function using the new harmonic parameter and the two types of activation functions, fuses all features according to the new harmonic function, obtains a new fusion result, and determines a mapping relationship between the electroencephalogram signal and a noise category corresponding to the other noise distribution according to the new fusion result. The harmonic parameter and the harmonic function are adaptively changed to realize adaptive judgment and reduce human interference.

[0143] A classification processing method of an electroencephalogram signal corresponding to the above embodiment, Figure 4 A structure block diagram of a classification processing device of an electroencephalogram signal provided by the fourth embodiment of the application is shown, and the classification processing device is applied to Figure 1 a server in the related art. The computer device corresponding to the server is connected to a corresponding database to obtain corresponding electroencephalogram data in the database. The computer device can also be connected to a corresponding client, and the client can also collect corresponding electroencephalogram data for the server. For convenience of description, only parts related to the embodiments of the application are shown.

[0144] Referring to Figure 4 , the classification processing device comprises:

[0145] The signal preprocessing module 41 is configured to obtain an electroencephalogram signal with a time sequence attribute, split the electroencephalogram signal according to a preset time length, determine N sub-signals after splitting, and N is an integer greater than zero.

[0146] The feature extraction module 42 is configured to extract power spectral density of each sub-signal on a preset frequency band respectively, obtain N power spectral densities, convert the electroencephalogram signal into an electroencephalogram time-frequency diagram according to the preset frequency band, the N power spectral densities and a time sequence relationship of the N sub-signals, extract noise features of the electroencephalogram time-frequency diagram, and obtain feature information representing noise.

[0147] The feature fusion module 43 is configured to construct a harmonic function based on a preset harmonic parameter and at least two types of activation functions, fuse all feature information using the harmonic function, and obtain a fusion result.

[0148] The classification processing module 44 is configured to determine that the brain electrical signal belongs to a noise category corresponding to a preset noise distribution when it is detected that the fusion result meets a preset condition.

[0149] Optionally, the signal preprocessing module 41 comprises:

[0150] The sliding unit is configured to slide along the time sequence of the brain electrical signal according to a preset sliding step using a sliding window with a preset length.

[0151] The cutting unit is configured to determine that the signal in the sliding window at each sliding stop is a cut sub-signal, and obtain N cut sub-signals.

[0152] Optionally, the classification processing device further comprises:

[0153] The first preprocessing unit is configured to select the brain electrical signal using a preset channel after obtaining the brain electrical signal with a time sequence attribute, and obtain a selected brain electrical signal.

[0154] The second preprocessing unit is configured to down-sample the selected brain electrical signal, and obtain a down-sampled brain electrical signal.

[0155] The signal preprocessing module 41 comprises:

[0156] The third preprocessing unit is configured to cut the down-sampled brain electrical signal according to a preset length.

[0157] Optionally, the feature extraction module 42 comprises:

[0158] The power spectral density acquisition unit is configured to perform discrete Fourier transform on each sub-signal in a preset frequency band, and obtain a power spectral density of the corresponding sub-signal as a transform result.

[0159] Optionally, the feature extraction module 42 comprises:

[0160] The feature extraction unit is configured to input the brain electrical time-frequency graph into a time-frequency feature extraction model, and output feature information representing noise.

[0161] The time-frequency feature extraction model comprises a convolution layer, a bidirectional long short-term memory layer, and a long short-term memory layer, and the feature extraction unit comprises:

[0162] The convolution sub-unit is configured to input the brain electrical time-frequency graph into the convolution layer, and output a convolution result.

[0163] The feature extraction sub-unit is configured to input the convolution result into the bidirectional long short-term memory layer, output a preliminary feature extraction result, input the preliminary feature extraction result into the long short-term memory layer, and output the feature information representing noise.

[0164] Optionally, the feature fusion module 43 comprises:

[0165] a variance obtaining unit configured to obtain a noise variance of the preset noise distribution;

[0166] a harmonic parameter determining unit configured to calculate a standard deviation of the noise variance, perform normalization on the standard deviation, and determine a result of the normalization as the harmonic parameter;

[0167] a weight value determining unit configured to obtain a Softplus function and a ReLU function, take the harmonic parameter as a weight value of the Softplus function, and take a difference between the harmonic parameter and 1 as a weight value of the ReLU function;

[0168] a harmonic function determining unit configured to take a weighted sum result of the Softplus function and the ReLU function as the harmonic function.

[0169] It should be noted that the information interaction between the above modules, the execution process and the like, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by them can be referred to the method embodiments part, and will not be repeated here.

[0170] Figure 5 A structural schematic diagram of a computer device provided in Embodiment Five of the present application is shown in FIG. 5. Figure 5 As shown in FIG. 5, the computer device of this embodiment comprises at least one processor (only one processor is shown in the figure), a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor implements the steps in any of the above-mentioned brain electrical signal classification processing method embodiments when executing the computer program. Figure 5

[0171] The computer device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that Figure 5 the computer device is only an example of the computer device and does not constitute a limitation on the computer device. The computer device can include more or fewer components than those shown, or combine certain components, or different components, for example, it can also include a network interface, a display screen, and an input device, etc.

[0172] ​The processor can be a CPU, and can also be other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0173] The memory includes a readable storage medium, an internal memory, etc., where the internal memory can be a memory of the computer device, and the internal memory provides an environment for running the operating system and the computer-readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the computer device, and in other embodiments, can also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory can include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium at least includes any entity or device that can carry computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, computer readable medium cannot be electrical carrier signal and telecommunication signal.

[0175] The above embodiment methods can also be implemented by a computer program product, which can be run on a computer device to make the computer device execute the steps of the above method embodiments.

[0176] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0177] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0178] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0179] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0180] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of classifying electroencephalographic signals, characterized by, The classification processing method comprises: Obtaining an electroencephalogram signal with time sequence attributes, dividing the electroencephalogram signal according to a preset time length, and determining N divided sub-signals, N being an integer greater than zero; Extracting power spectrum density of each sub-signal on a preset frequency band to obtain N power spectrum densities, converting the electroencephalogram signal into an electroencephalogram time-frequency graph according to the preset frequency band, the N power spectrum densities and a time sequence relationship of the N sub-signals, extracting noise features of the electroencephalogram time-frequency graph to obtain feature information representing noise; Based on a preset harmonic parameter and at least two types of activation functions, a harmonic function is constructed, and all feature information is fused using a full connection layer corresponding to the harmonic function to obtain a fusion result, and the preset harmonic parameter is constructed according to a preset noise distribution; When it is detected that the fusion result meets a preset condition, it is determined that the electroencephalogram signal belongs to a noise category corresponding to the preset noise distribution; Extracting noise features of the electroencephalogram time-frequency graph to obtain feature information representing noise comprises: Inputting the electroencephalogram time-frequency graph into a time-frequency feature extraction model to output feature information representing noise; Wherein, the time-frequency feature extraction model comprises a convolution layer, a bidirectional long short-term memory layer and a long short-term memory layer, and the electroencephalogram time-frequency graph is input into the time-frequency feature extraction model to output feature information representing noise, which comprises: Inputting the electroencephalogram time-frequency graph into the convolution layer to output a convolution result; Inputting the convolution result into the bidirectional long short-term memory layer to output a preliminary feature extraction result, and inputting the preliminary feature extraction result into the long short-term memory layer to output feature information representing noise, wherein the convolution layer is a three-layer one-dimensional convolution layer, the long short-term memory layer and the bidirectional long short-term memory layer are each one layer, and the time-frequency feature extraction model is jointly trained with a subsequent full connection layer.

2. The classification processing method according to claim 1, characterized by, Dividing the electroencephalogram signal according to a preset time length to determine N divided sub-signals comprises: Using a preset time length sliding window, sliding along the time sequence of the electroencephalogram signal according to a preset sliding step; Determining the signal in the sliding window at each sliding stop as a divided sub-signal to obtain N divided sub-signals.

3. The classification processing method according to claim 1, characterized by, After obtaining the electroencephalogram signal with time sequence attributes, it further comprises: Selecting the electroencephalogram signal using a preset channel to obtain a selected electroencephalogram signal; Down-sampling the selected electroencephalogram signal to obtain a down-sampled electroencephalogram signal; Dividing the electroencephalogram signal according to a preset time length comprises: Dividing the down-sampled electroencephalogram signal according to a preset time length.

4. The classification processing method according to claim 1, characterized by, Extracting power spectrum density of each sub-signal on a preset frequency band comprises: Performing discrete Fourier transform on each sub-signal on a preset frequency band to obtain a transformation result as the power spectrum density of the corresponding sub-signal.

5. The classification processing method according to claim 1, characterized by, Based on a preset harmonic parameter and at least two types of activation functions, constructing a harmonic function comprises: Obtaining noise variance of a preset noise distribution; Calculating the standard deviation of the noise variance, normalizing the standard deviation, and determining the normalized result as a harmonic parameter; obtaining a Softplus function and a ReLU function, taking the harmonic parameter as a weight value of the Softplus function, and taking a difference between the harmonic parameter and 1 as a weight value of the ReLU function; performing a weighted summation of the Softplus function and the ReLU function to obtain a harmonic function.

6. An apparatus for classifying electroencephalographic signals, characterized by The classification processing apparatus comprises: a signal preprocessing module, configured to obtain an electroencephalogram signal with a time sequence attribute, split the electroencephalogram signal according to a preset time length, and determine N split sub-signals, N being an integer greater than zero; a feature extraction module, configured to extract a power spectral density of each sub-signal in a preset frequency band to obtain N power spectral densities, convert the electroencephalogram signal into an electroencephalogram time-frequency graph according to the preset frequency band, the N power spectral densities, and a time sequence relationship of the N sub-signals, extract a noise feature from the electroencephalogram time-frequency graph to obtain feature information representing noise; a feature fusion module, configured to construct a harmonic function based on a preset harmonic parameter and at least two types of activation functions, fuse all feature information using a full connection layer corresponding to the harmonic function to obtain a fusion result, and construct the preset harmonic parameter according to a preset noise distribution; a classification processing module, configured to determine that the electroencephalogram signal belongs to a noise category corresponding to the preset noise distribution when it is detected that the fusion result meets a preset condition. The feature extraction module comprises: a feature extraction unit, configured to input the electroencephalogram time-frequency graph into a time-frequency feature extraction model, and output feature information representing noise; wherein the time-frequency feature extraction model comprises a convolution layer, a bidirectional long short-term memory layer, and a long short-term memory layer, and the output of the feature information representing noise comprises: a convolution sub-unit, configured to input the electroencephalogram time-frequency graph into the convolution layer, and output a convolution result; a feature extraction sub-unit, configured to input the convolution result into the bidirectional long short-term memory layer, output a preliminary feature extraction result, input the preliminary feature extraction result into the long short-term memory layer, and output the feature information representing noise, wherein the convolution layer is a three-layer one-dimensional convolution layer, the long short-term memory layer and the bidirectional long short-term memory layer are each one layer, and the time-frequency feature extraction model is jointly trained with a subsequent full connection layer.

7. The classification processing device according to claim 6, characterized by The signal preprocessing module comprises: a sliding unit, configured to use a sliding window with a preset time length, and slide along a time sequence of the electroencephalogram signal according to a preset sliding step; a splitting unit, configured to determine a signal in the sliding window at each sliding stop as a split sub-signal, and obtain N split sub-signals.

8. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the classification processing method of any one of claims 1 to 5 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the classification processing method of any one of claims 1 to 5.

Images