Throat electrical signal classification method and device based on neural architecture search and electronic equipment
Through neural architecture search technology, the network architecture is optimized in the factorization and layered search space, combined with the extended long-term and short-term memory network and attention mechanism, the limitations of model structure design in traditional laryngeal signal classification are solved and the classification of laryngeal signal is improved. Accuracy has promoted the progress of research on electroporation signals.
Patent Information
- Application Number
- CN202411162246.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-12
AI Technical Summary
In the classification of power laryngeal signal, traditional long-term memory networks have problems such as limited memory unit capacity of controllers, excessive search space range, and limited diversity of network models, which limit their performance in capturing long-term span dependence and improving classification accuracy.
Using a neural architecture search method, the network architecture is automatically designed and optimized in the factorization and layered search space through the controller to generate better model structures, combined with the extended long and short-term memory network and attention mechanism, optimize the model structure to improve Classification accuracy of throat signal.
The workload of artificially designed network architecture is reduced, the model structure is optimized, the accuracy of laryngeal signal classification is significantly improved, and new progress in the research and application of laryngeal signal is promoted.
Smart Images

Figure CN118940141A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of bioelectric signal classification, and in particular to a laryngeal electrical signal classification method, device and electronic device based on neural architecture search. Background Art
[0002] Interpreting the electrical signals generated by laryngeal muscle movements is of great significance for achieving speech recognition and communication. In order to quickly and accurately extract the semantic information of laryngeal electrical signals, laryngeal electrical signal interpretation based on deep learning theory has become a major technology. Among them, the laryngeal electrical signal interpretation technology based on long short-term memory networks has achieved good performance. However, the controller memory unit capacity of traditional long short-term memory networks is limited, which limits its performance in capturing long-term span dependency problems, and the search space is too large and the network model diversity is limited. To this end, based on the advantage of neural architecture search theory that can automatically design and optimize the network architecture in the search space to find a better model structure, conducting laryngeal electrical signal classification based on neural architecture search is a new paradigm that is expected to break through the bottleneck of laryngeal electrical signal classification network design based on traditional long short-term memory networks, so as to improve the classification accuracy of laryngeal electrical signals. Summary of the invention
[0003] The present application provides a laryngeal electrical signal classification method, device, medium and electronic device based on neural architecture search, which can realize laryngeal electrical signal feature extraction and classification in combination with neural architecture search technology, and promote the research on laryngeal electrical signals.
[0004] In a first aspect, the present application provides a laryngeal electrical signal classification method based on neural architecture search, comprising:
[0005] Acquire and preprocess laryngeal electrical signal data, and divide the preprocessed laryngeal electrical signal data into a training set, a validation set, and a test set;
[0006] generating a sub-model by a controller, wherein the controller is a model that performs a neural architecture search method;
[0007] Obtaining a small batch training set from the training set to train the sub-model, and using the validation set to determine the classification accuracy of the trained sub-model;
[0008] Using the classification accuracy as a reward of the controller, updating the controller through the reward, and regenerating a sub-model through the updated controller, and using the sub-model with the highest classification accuracy as a laryngeal electrical signal classification model;
[0009] The laryngeal electrical signal classification model is trained using the training set, and the laryngeal electrical signal classification model after training is used to classify the laryngeal electrical signal data in the test set.
[0010] According to the laryngeal electrical signal classification method based on neural architecture search in this embodiment, NAS is used to design a classification model for laryngeal electrical signals, which can reduce the workload of manually designing the network architecture; and by updating the NAS controller and optimizing the model structure of its design, the model structure can be optimized to the greatest extent, which is beneficial to improving the classification accuracy of the classification model for laryngeal electrical signals; it provides new ideas for the current laryngeal electrical signal research field and can promote the research and application of laryngeal electrical signal classification.
[0011] In an exemplary embodiment, based on the above method, a laryngeal electrical signal preprocessing method comprises:
[0012] The laryngeal electroacoustic signals were filtered by a sixth-order Butterworth bandpass filter;
[0013] Down-sampling the filtered laryngeal electrical signal;
[0014] Baseline correction was performed on the down-sampled laryngeal electrical signals;
[0015] The baseline-corrected laryngeal electrical signal was subjected to universal average reference to remove common-mode noise and obtain clean laryngeal electrical signal data.
[0016] In an exemplary embodiment, based on the above method, the generating of the sub-model by the controller includes:
[0017] Determining that the search space of the controller is a factorized hierarchical search space, wherein the factorized hierarchical search space includes a plurality of network components, including convolution types such as mobile reverse bottleneck convolution and depthwise separable convolution, and convolution kernel sizes and connection methods used;
[0018] The controller determines the layer structure of each network component in the factorized hierarchical search space to obtain a sub-model.
[0019] In an exemplary embodiment, based on the foregoing method, after the convolutional neural network units in the search space are connected by the controller, the method further includes:
[0020] A channel attention residual structure module and a spatial attention residual structure module based on serial connection are added after the network components output by the controller.
[0021] In an exemplary embodiment, based on the above method, obtaining a small batch training set from the training set to train the sub-model includes:
[0022] Inputting the laryngeal electrical signals included in the small batch training set into the sub-model to obtain the predicted labels output by the sub-model;
[0023] Determine the cross entropy loss between the sample labels of the laryngeal electrical signals in the small batch training set and the predicted labels;
[0024] The expected loss is determined by using a Monte Carlo estimation algorithm, and the gradient information corresponding to the loss is obtained by back propagation, and the parameters of the sub-model are updated using the gradient information;
[0025] Repeat the above steps to obtain the trained sub-model.
[0026] In an exemplary embodiment, based on the above method, the using the validation set to determine the classification accuracy of the trained sub-model includes:
[0027] The sub-model trained with the small batch training set is verified using the validation set data to obtain the classification accuracy of the sub-model.
[0028] In an exemplary embodiment, based on the above method, taking the classification accuracy as a reward of the controller and updating the controller by the reward includes:
[0029] The parameter gradient of the controller corresponding to the reward is calculated by a policy gradient algorithm, the parameter gradient is input into an optimizer, and the parameters of the controller are updated by the optimizer.
[0030] In an exemplary embodiment, based on the above method, the sub-model with the highest classification accuracy is used as the laryngeal electrical signal classification model, including:
[0031] The sub-model with the highest classification accuracy was selected as the laryngeal electrical signal classification model and retrained using the complete training set.
[0032] In an exemplary embodiment, based on the above method, the step of acquiring laryngeal electrical signal data and preprocessing the data includes:
[0033] Step 1: Perform band-pass filtering on the laryngeal electrical signal through a sixth-order Butterworth band-pass filter to obtain a target frequency band of 0.5 to 60 Hz for the laryngeal electrical signal;
[0034] Step 2: Downsample the filtered laryngeal electrical signal to 200 Hz;
[0035] Step 3: Perform baseline correction on the laryngeal electrical signal after downsampling, determine the correction window for calculating the baseline value in the laryngeal electrical signal to be 100ms before the channel, calculate the average value of the laryngeal electrical signal within this time window, and subtract the calculated average value from each time point of the signal;
[0036] Step 4: Use the baseline-corrected laryngeal electrical signal to obtain a universal average reference signal. Specifically, by calculating the average potential of all recording electrodes and using it as the potential of the reference electrode, subtracting the potential of each electrode to obtain the universal average reference signal as the preprocessed laryngeal electrical signal data.
[0037] In an exemplary embodiment, based on the aforementioned method, the controller is an extended long short-term memory network composed of a scalar long short-term memory network and a matrix long short-term memory network; each scalar long short-term memory unit is responsible for generating short-term dependency operations, and each matrix long short-term memory unit is responsible for generating long-term dependency operations; the extended long short-term memory network uses the two units alternately, considering local and global information at each step, and generating a more effective sub-model architecture.
[0038] In a second aspect, the present application provides a laryngeal electrical signal classification device based on neural architecture search, comprising:
[0039] A data acquisition module, used to acquire and preprocess laryngeal electrical signal data, and divide the preprocessed laryngeal electrical signal data into a training set, a validation set, and a test set; a sub-model construction module, used to generate a sub-model through a controller, wherein the controller is a model for executing a neural architecture search method;
[0040] A sub-model optimization module, used to obtain a small batch data set from the training set to train the sub-model, and use the validation set to determine the classification accuracy of the trained sub-model;
[0041] A controller optimization module, used for taking the classification accuracy as a reward of the controller, updating the controller by the reward, and regenerating a sub-model by the updated controller, and taking the sub-model with the highest classification accuracy as a laryngeal electrical signal classification model;
[0042] The classification model determination module is used to train the laryngeal electrical signal classification model through the training set, and use the trained laryngeal electrical signal classification model to classify the laryngeal electrical signal data in the test set.
[0043] In a third aspect, the present application provides an electronic device, the electronic device comprising a memory and one or more processors. The memory stores one or more computer programs, the computer programs comprising instructions, and when the instructions are executed by the processor, the electronic device can execute the laryngeal electrical signal classification method based on neural architecture search as in the first aspect.
[0044] In a fourth aspect, the present application provides a computer-readable medium having instructions stored therein, which, when executed on an electronic device, enables the electronic device to execute the laryngeal electrical signal classification method based on neural architecture search as in the first aspect.
[0045] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the laryngeal electrical signal classification method based on neural architecture search as described in the first aspect.
[0046] It can be understood that the beneficial effects that can be achieved by the laryngeal electrical signal classification device, electronic device, computer-readable medium, and computer program product based on neural architecture search provided above can refer to the beneficial effects in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A schematic diagram of a flow chart of a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0048] Figure 2 A schematic diagram of a factorized hierarchical search space in a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0049] Figure 3 A schematic diagram of the controller structure in the laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0050] Figure 4 A schematic diagram of a sub-model structure in a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0051] Figure 5 A schematic diagram of a channel attention mechanism in a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0052] Figure 6 A schematic diagram of a spatial attention mechanism in a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application;
[0053] Figure 7 A schematic diagram of the structure of a laryngeal electrical signal classification device based on neural architecture search provided in an embodiment of the present application;
[0054] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0055] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the difference. It should be noted that in the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more.
[0056] It should be noted that the “at…” in the embodiments of the present application may be the instant when a certain situation occurs, or may be a period of time after the occurrence of a certain situation, and the embodiments of the present application do not specifically limit this.
[0057] The implementation of this embodiment will be described in detail below with reference to the accompanying drawings.
[0058] This embodiment first provides a laryngeal electrical signal classification method based on neural architecture search. Exemplarily, the laryngeal electrical signal classification method based on neural architecture search can be applied to various electronic devices such as computers (PCs), tablets, virtual reality / augmented reality devices, wearable devices, industrial computers, and car computers; it can also be applied to servers, cloud computing, server clusters, etc., and this embodiment does not make any special limitations on this.
[0059] Figure 1 A schematic flow chart of a laryngeal electrical signal classification method based on neural architecture search provided in an embodiment of the present application is shown.
[0060] like Figure 1 As shown, the laryngeal electrical signal classification method based on neural architecture search may include the following steps:
[0061] Step 101: Acquire laryngeal electrical signal data and perform preprocessing, and divide the preprocessed laryngeal electrical signal data into a training set, a validation set, and a test set.
[0062] The laryngeal electrical signals with sample labels are obtained from the public data set. Since the original laryngeal electrical signals contain interference components such as physiological activities and environmental noise, which affect the subsequent classification results, the laryngeal electrical signals need to be preprocessed before entering the classification network to make the effective signal information clearer and more obvious. Specifically, the laryngeal electrical signal data is obtained and preprocessed, including the following:
[0063] Step 1: Perform bandpass filtering on the laryngeal electrical signal through a sixth-order Butterworth bandpass filter to obtain the target frequency band of 0.5-60 Hz for the laryngeal electrical signal. This filtering process can effectively filter out signals outside the target frequency, such as high-frequency electrical signals generated by physiological activities such as vomiting.
[0064] Step 2: Downsample the filtered laryngeal electrical signal to 200 Hz. Downsampling the filtered laryngeal electrical signal to 200 Hz can reduce the amount of data and the amount of calculation, while also saving resources during transmission and storage.
[0065] Step 3: Perform baseline correction on the laryngeal electrical signal after downsampling, determine the correction window for calculating the baseline value in the laryngeal electrical signal as the first 100ms of the channel, calculate the average value of the laryngeal electrical signal within this time window, and subtract the calculated average value from each time point of the signal. Specifically, the time window of 100ms before the channel is used as the correction window, calculate the average value of the laryngeal electrical signal within this time window, use the average value as the baseline value, subtract the average value from the original value of the laryngeal electrical signal at each time point, and obtain the laryngeal electrical signal after baseline correction. The laryngeal electrical signal after downsampling eliminates signal drift through baseline correction, subtracts the average value of the first 100ms data of the corresponding channel, and reduces the influence of signal drift on the laryngeal electrical signal.
[0066] Step 4: Use the baseline-corrected laryngeal electrical signal to obtain a universal average reference signal. Specifically, the average potential of all recording electrodes is calculated and used as the potential of the reference electrode. The potential of each electrode is subtracted to obtain a universal average reference signal as the preprocessed laryngeal electrical signal data. The common noise of all channels is eliminated by subtracting the average value of the corresponding position of all channel data from the universal average reference, thereby obtaining a pure laryngeal electrical signal, which is conducive to subsequent processing.
[0067] Step 102: Generate a sub-model through a controller, where the controller is a model that executes a neural architecture search method. The controller can search for network components in the search space to generate a network structure, and the network structure generated by the controller is called a sub-model. The controller is an extended long short-term memory network (Extended Long Short-Term Memory, xLSTM) composed of a scalar long short-term memory network (Scalar LSTM, sLSTM) and a matrix long short-term memory network (Matrix LSTM, mlSTM). Each scalar long short-term memory unit is responsible for generating short-term dependency operations, and each matrix long short-term memory unit is responsible for generating long-term dependency operations; the extended long short-term memory network uses two units alternately, considers local and global information at each step, and generates a more effective sub-model architecture.
[0068] Generating a sub-model through a controller specifically includes: determining a factorized hierarchical search space of the controller, wherein the factorized hierarchical search space includes a plurality of network components, which may be: a mobile reverse bottleneck convolution and a depthwise separable convolution and the convolution kernel size and connection method adopted; determining the layer structure of each of the network components in the factorized hierarchical search space through the controller to obtain a sub-model.
[0069] Exemplarily, a channel attention residual structure module and a spatial attention residual structure module based on serial connection are added after the network components output by the controller to obtain a sub-model.
[0070] At present, the research field of bioelectric signals based on neural architecture search is mainly concentrated on EEG, especially in the field of EEG emotion classification, and good results have been achieved. However, the application of weak surface electromyographic signals such as laryngeal electrical signals has not been widely carried out, and there is a lack of relevant experience in this field. This method extends the neural architecture search to the field of laryngeal electrical signal feature extraction, adds a factorized hierarchical search space, and overcomes the problem that the original search space is too large and the diversity of network layers is limited. In the search strategy of neural architecture search, the network component units are searched from the search space through a controller based on an extended long short-term memory network (xLSTM), thereby connecting to a complete sub-model, and the controller is continuously updated through reinforcement learning to generate a sub-model with the highest classification accuracy. Since the present invention introduces neural architecture search, the amount of manual intervention is less than the traditional network design method, and the diversity of the component structure layers is taken into account, so the generated network has good accuracy and low latency.
[0071] In this embodiment, NAS uses a factorized hierarchical search space, such as Figure 2As shown in the figure, a convolutional neural network model is divided into a specified number of network components. Each network component can include multiple layers, so the deep and shallow model structures of different parts of the network can be designed. The search space of the controller is concentrated in the sub-search space in each network component. The sub-search space in each network component includes:
[0072] (1) Convolution operation mode: including ordinary convolution (Conv), depthwise separable convolution (DepthwiseSeparableConv), moving reverse bottleneck convolution (MBConv), etc.
[0073] (2) Convolution kernel size: 1×1, 3×3, 5×5.
[0074] (3) SERatio: 0, 0.1, 0.25, 0.5. SERatio is the scaling factor in the channel attention mechanism.
[0075] (4) Skip operation: pooling, residual connection, dense connection or no skipping.
[0076] (5) Output layer convolution kernel size F i .
[0077] (6) The number of layers N of each network component unit i , this number of layers refers to the number of repeated layers in each component unit of the neural network.
[0078] The controller is an xLSTM model composed of alternating sLSTM and mLSTM units, each of which is responsible for generating a specific component of the sub-model. Each sLSTM unit is responsible for generating short-term dependency operations, such as convolution kernel size, convolution type, etc. Each mLSTM unit is responsible for generating long-term dependency operations, such as skip connections, SE ratio, filter size, etc. By alternating sLSTM and mLSTM units, local and global information can be considered at each step, generating more complex and effective sub-model architectures. Figure 3 As shown, through the hidden state h of the previous time step t-1 and the current input x t To select the current convolution operation type, and use the selection result as the input of the next time step to select the convolution kernel size, send the selection result to the next time step to select whether to use the Squeeze-Excitation (SE) ratio, and then send the selection result and the hidden state of the current time step output to the next time step, and select the cross-layer connection method; the next time step selects the output layer convolution kernel size F i; The subsequent time steps select the number of layers in each network component unit. A total of T time steps are executed to generate a specified number of network components to form the entire sub-model.
[0079] The above steps will be repeated until all the network components of the convolutional neural network are predicted. The controller connects these generated network components to form a convolutional network, i.e., a sub-model.
[0080] In LSTM, h t-1 Contains the knowledge that the controller has learned so far, memorizing the information of the previous time steps, x t The output from the previous time step. t-1 and c t To represent the memory state, the forget gate f can be calculated t , input gate i t , the unit state update value z t And the output gate o t , as shown in Table 1:
[0081] Table 1
[0082]
[0083]
[0084] Compared with LSTM, sLSTM allows information sharing and transmission between multiple memory units, realizing information transmission between different time steps. sLSTM introduces exponential activation functions into the input gate and forget gate to more effectively update their internal states, giving LSTM the ability to modify storage decisions. It also introduces normalization and stabilization. By introducing a normalized state, the product of the input gate and all future forget gates is added, which can better handle information flow, especially in scenarios where storage decisions need to be revised. The introduction of an exponential activation function may lead to larger values, resulting in overflow, so an additional stabilization state m is used. t To stabilize the gating signals of the input gate and the forget gate. And sLSTM can have multiple memory cells like the original LSTM. Multiple memory cells are connected by a loop from the hidden state vector h to the memory cell input z and the gate (input gate i, forget gate f, output gate o). z 、r i 、r f 、r o Implement memory mixing. These recurrent connections allow the model to pass information between different time steps, thereby better capturing long-term dependencies. The new sLSTM can have multiple heads. The introduction of sLSTM heads and the combination of exponential gating provide a new approach to memory mixing. The sLSTM unit update rules are as shown in Table 2:
[0085] Table 2
[0086]
[0087] Based on the original LSTM, mLSTM increases the memory unit of LSTM from scalar c∈R to matrix C∈R in order to improve the storage capacity. d×d . And the covariance update rule is introduced, so that mLSTM can be fully parallelized. Introduce q (query), k (key), and v (value) in the Transformer mechanism. When information needs to be retrieved from a memory unit, the query vector q is used to find the key vector k that matches it, and then return the corresponding value vector v. By introducing the q, k, and v mechanism, the mLSTM memory unit can store and retrieve more and more complex information. The use of the query vector q can quickly and accurately retrieve relevant information, improving the overall performance of the model. This design enables mLSTM to process complex time series data more efficiently and has significant advantages in information storage and retrieval. The covariance update rule is as follows:
[0088]
[0089] The covariance update rule is equivalent to the Fast Weight Programmers. The latter introduces a constant decay rate and a constant learning rate in the update process. The covariance update rule is essentially similar to the mechanism of the Fast Weight Programmer and is used to dynamically adjust the model parameters. Based on this idea, the covariance update rule is integrated into the LSTM framework, where the forget gate corresponds to the decay rate, the input gate corresponds to the learning rate, and the output gate is used to adjust the size of the retrieved vector. The forward pass of mLSTM is shown below, where C t is the matrix memory cell state, v t and k t are the value vector and the key vector, q t is the query vector, as shown in Table 3:
[0090] Table 3
[0091]
[0092] where w i ,w f ,w c, w o represents the weight matrix in the network, b i ,b f ,b c ,b o Represents the bias parameter of the network, by updating c t With ht , h t Input into the dense nested network to get the input value x of the next state t+1 .
[0093] The search space in this embodiment is a factorized hierarchical search space. Compared with the traditional technology of building a network by searching a network block and then stacking the network blocks into a network, the use of a factorized hierarchical search space overcomes the problem of the traditional search space having too large a search range. The hierarchical search structure takes into account the diversity of each layer and the limited nature of the search space.
[0094] After the convolutional neural network units in the search space are connected by the controller, the method further includes: adding an attention mechanism module (CBAM), a global average pooling layer (GAP) and a fully connected layer (FC) of a convolutional module after the network component unit output by the controller to obtain a sub-model, as shown in the attached figure. Figure 4 shown.
[0095] In order to improve the performance and robustness of the laryngeal electrical signal classification model and make full use of the channel and spatial features of the laryngeal electrical signal, the CBAM (convolutional attention) module is introduced. The CBAM module solves the problem that traditional convolutional neural networks do not pay enough attention to key features when processing complex physiological signals. By introducing the channel attention and spatial attention mechanisms, it achieves enhanced feature selectivity and noise suppression, significantly improving the classification performance of the model and its ability to capture key features. The channel attention mechanism structure is as follows: Figure 5 As shown:
[0096] For the input feature map, first perform global maximum pooling (maxpool) and global average pooling (avgpool) operations on each channel to calculate the maximum eigenvalue and average eigenvalue on each channel. Generate two vectors containing the number of channels, representing the global maximum feature and average feature of each channel respectively. The feature vectors after global maximum pooling and average pooling are input into a shared fully connected layer. This fully connected layer is used to learn the attention weights for each channel. Through learning, the network can adaptively decide which channels are more important for the current task. Intersect the global maximum eigenvector and the average eigenvector to obtain the final attention weight vector. In order to ensure that the attention weights are between 0 and 1, a Sigmoid activation function is applied to generate channel attention weights, which will be applied to each channel of the original feature map. Using the obtained attention weights, multiply them by each channel of the original feature map to obtain the attention-weighted channel feature map. This will emphasize the channels that are helpful for the current task and suppress irrelevant channels.
[0097] The structure diagram of the spatial attention mechanism is as follows Figure 6 As shown:
[0098] For the input feature map, max pooling and average pooling operations are performed along the channel dimension to generate features of different context scales. The max pooled and average pooled features are concatenated along the channel dimension to obtain a feature map with context information of different scales. This feature map is then processed through a convolutional layer to generate spatial attention weights. Similar to the channel attention module, a sigmoid activation function is applied to the generated spatial attention weights to limit the weights between 0 and 1. The resulting spatial attention weights are applied to the original feature map to weight the features at each spatial position.
[0099] In order to reduce the parameters passed to the final softmax classifier, a global average pooling layer and a fully connected layer are added at the end of the network structure, and finally a classification network structure is built.
[0100] Step 103: Obtain a small batch training set from the training set to train the sub-model, and use the validation set to determine the classification accuracy of the trained sub-model.
[0101] Train the sub-model generated by the controller. After the controller generates a network structure, it only provides hyperparameters such as the network's learning rate, number of convolutional layers, convolution kernel size, and pooling layer size. The weight parameters in the network model are not provided, so the sub-model needs to be trained with a small batch training set of samples prepared in advance. The training process is as follows:
[0102] A plurality of small-batch training sets are divided from the training set, and the laryngeal electrical signals included in the small-batch training set are input into the sub-model to obtain the predicted labels output by the sub-model; the cross entropy loss between the sample labels of the laryngeal electrical signals in the small-batch training set and the predicted labels is determined; the expected loss is obtained by using a Monte Carlo estimation algorithm, and the gradient information corresponding to the expected loss is determined by back propagation, and the parameters of the sub-model are updated using the gradient information; the above steps are repeated to obtain the trained sub-model.
[0103] A part of the samples in the training set is extracted as a small batch training set, and the sub-model parameters are continuously optimized after back propagation and gradient update. According to the sample labels of the laryngeal electrical signals in the small batch training set and the predicted labels of the sub-model prediction output, the cross entropy loss of the sub-model is calculated: L(m; ω), where m is the sub-model generated by the controller and ω is the shared parameter of the sub-model.
[0104] Then use the momentum gradient descent algorithm to minimize the expected loss E m~π[L(m; ω)], the gradient information required for gradient update is calculated by standard back propagation, and the Monte Carlo estimation method is used to calculate the expected loss gradient information:
[0105]
[0106] m i is a sub-model sampled from the controller, and M represents the number of samples sampled each time when using the Monte Carlo method to approximate the expected loss gradient. In the design, M is set to 1, and the sub-model shared parameter ω is updated through the gradient information in the model to obtain the best performance. In this process, the controller parameter θ c Fixed, only the sub-model parameters are optimized to avoid the impact of controller parameter changes on the network architecture in the sub-model.
[0107] When the loss is less than the preset threshold, the training is completed. After the sub-model is trained with a small batch training set, the trained sub-model is verified on the validation set to obtain the classification accuracy R of the sub-model on the validation set.
[0108] Step 104: The classification accuracy is used as a reward for the controller, the controller is updated by the reward, and a sub-model is generated again by the updated controller, and the sub-model with the highest classification accuracy is used as the laryngeal electrical signal classification model.
[0109] This embodiment adopts a reinforcement learning strategy to optimize the controller. Specifically: the parameter gradient of the controller corresponding to the reward is calculated by a policy gradient algorithm, the parameter gradient is input into an optimizer, and the parameters of the controller are updated by the optimizer.
[0110] Use the classification accuracy R as a reward to feed into the controller to update the controller parameters θ c The network structure predicted by the controller can be viewed as an action designed for the sub-model architecture. 1:T Each action predicted by the controller is considered as an action, denoted as a i , T is the number of actions that the controller must predict in order to design the neural network architecture. This classification accuracy R is used as the reward signal and reinforcement learning is used to train the controller. To find the optimal architecture, the controller maximizes its expected reward, given by J(θ c )express:
[0111]
[0112] Since the reward signal R is not differentiable, a policy gradient method is needed to iteratively update the gradient information:
[0113]
[0114] The calculated gradient information is then input into the Adam optimizer to update the controller parameters:
[0115]
[0116] In the controller parameter θ c When updating, the sub-model parameters are fixed to prevent the sub-model parameter update from affecting the gradient calculation of the controller parameters.
[0117] Repeat the above process, continuously generate new sub-models for training and verification, and obtain the accuracy R as reward feedback to the controller. Update the controller parameters through policy gradient and optimizer. When the expected return J(θ c ) is greater than the preset value, it can be determined that the controller training is completed, so as to find the best network structure as the laryngeal electrical signal classification model.
[0118] Step 105: training the laryngeal electrical signal classification model using the training set, and using the trained laryngeal electrical signal classification model to classify the laryngeal electrical signals of the test set.
[0119] The trained laryngeal electrical signal classification model can be used to extract features and classify biological laryngeal electrical signals, thereby enhancing the research and application of laryngeal electrical signals.
[0120] The best sub-model is selected from the generated sub-models, trained from scratch using the complete training set samples, and then tested using the test set samples to obtain the laryngeal electrical signal classification results. Alternatively, a certain number of laryngeal electrical signal samples are re-obtained to test the trained laryngeal electrical signal classification model to obtain the classification results of the laryngeal electrical signal classification model.
[0121] In this embodiment, the neural architecture search technology is applied to the classification of laryngeal electrical signals, which overcomes the problem that the traditional manual design of neural networks is labor-intensive and requires a lot of prior knowledge. The controller automatically searches for the required network components in the search space for combination, reduces the impact of human intervention, and further designs a laryngeal electrical signal classification network that meets the task requirements.
[0122] Furthermore, this embodiment also provides a laryngeal electrical signal classification device based on neural architecture search, which can be used to execute the above-mentioned laryngeal electrical signal classification method based on neural architecture search. Figure 7 As shown, the laryngeal electrical signal classification device 60 based on neural architecture search may specifically include the following:
[0123] A data acquisition module 61 is used to acquire a laryngeal electrical signal data set, and divide the laryngeal electrical signal data set into a training set, a validation set, and a test set; a sub-model construction module 62 is used to generate a sub-model through a controller, and the controller is a model that executes a neural architecture search method; a sub-model optimization module 63 is used to acquire a small batch training set from the training set to train the sub-model, and use the validation set to determine the classification accuracy of the trained sub-model; a controller optimization module 64 is used to use the classification accuracy as a reward for the controller, update the controller through the reward, and generate a sub-model again through the updated controller, and use the sub-model with the highest classification accuracy as the laryngeal electrical signal classification model; a classification model determination module 65 is used to train the laryngeal electrical signal classification model through the training set, and use the trained laryngeal electrical signal classification model to classify the laryngeal electrical signals of the test set.
[0124] The specific details of each module or unit in the above-mentioned laryngeal electrical signal classification device based on neural architecture search have been described in detail in the corresponding laryngeal electrical signal classification method based on neural architecture search, so they will not be repeated here.
[0125] The embodiment of the present application also provides an electronic device, Figure 8 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. Figure 8 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0126] like Figure 8 As shown, electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In RAM 603, various programs and data required for system operation are also stored. CPU 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.
[0127] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.
[0128] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, the above-mentioned functions defined in the embodiments of the present application are executed.
[0129] For example, when the computer program is executed by the central processing unit (CPU) 601, the following can be performed: obtaining a laryngeal electrical signal data set, dividing the laryngeal electrical signal data set into a training set and a validation set; generating a sub-model through a controller, wherein the controller is a model that executes a neural architecture search method; obtaining a small batch training set from the training set to train the sub-model, and using the validation set to determine the classification accuracy of the trained sub-model; using the classification accuracy as a reward for the controller, updating the controller through the reward, and generating a sub-model again through the updated controller, and using the sub-model with the highest classification accuracy as a laryngeal electrical signal classification model; training the laryngeal electrical signal classification model through the training set, and using the trained laryngeal electrical signal classification model to classify the laryngeal electrical signals of the test set.
[0130] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0131] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0132] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.
[0133] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device implements the method described in the above embodiment.
[0134] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0135] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. A method for laryngeal electrical signal classification based on neural architecture search, It is characterized in that include: Acquire and preprocess laryngeal electrical signal data, and divide the preprocessed laryngeal electrical signal data into a training set, a validation set, and a test set; generating a sub-model by a controller, wherein the controller is a model that performs a neural architecture search method; Obtaining a small batch training set from the training set to train the sub-model, and using the validation set to determine the classification accuracy of the trained sub-model; Using the classification accuracy as a reward of the controller, updating the controller through the reward, and regenerating a sub-model through the updated controller, and using the sub-model with the highest classification accuracy as a laryngeal electrical signal classification model; The laryngeal electrical signal classification model is trained using the training set, and the laryngeal electrical signal classification model after training is used to classify the laryngeal electrical signal data in the test set.
2. The method according to claim 1, It is characterized in that The generating of the sub-model by the controller includes: Determining that the search space of the controller is a factorized hierarchical search space, wherein the factorized hierarchical search space includes a plurality of network components, including a moving reverse bottleneck convolution and a depthwise separable convolution and a convolution kernel size and a connection method adopted; The controller determines the layer structure of each network component in the factorized hierarchical search space to obtain a sub-model.
3. The method according to claim 2, It is characterized in that Also includes: A channel attention residual structure module and a spatial attention residual structure module based on serial connection are added after the network components output by the controller.
4. The method according to claim 1, It is characterized in that The method of obtaining laryngeal electrical signal data and performing preprocessing includes: Step 1: Perform band-pass filtering on the laryngeal electrical signal through a sixth-order Butterworth band-pass filter to obtain a target frequency band of 0.5 to 60 Hz for the laryngeal electrical signal; Step 2: Downsample the filtered laryngeal electrical signal to 200 Hz; Step 3: Perform baseline correction on the laryngeal electrical signal after downsampling, determine the correction window for calculating the baseline value in the laryngeal electrical signal to be 100ms before the channel, calculate the average value of the laryngeal electrical signal within the correction window, and subtract the calculated average value from each time point of the signal; Step 4: Use the baseline-corrected laryngeal electrical signal to obtain a universal average reference signal. Specifically, by calculating the average potential of all recording electrodes and using it as the potential of the reference electrode, subtracting the potential of each electrode to obtain the universal average reference signal as the preprocessed laryngeal electrical signal data.
5. The method according to claim 1, It is characterized in that The controller is an extended long short-term memory network composed of a scalar long short-term memory network and a matrix long short-term memory network; each scalar long short-term memory unit is responsible for generating short-term dependency operations, and each matrix long short-term memory unit is responsible for generating long-term dependency operations; the extended long short-term memory network uses the two units alternately, considering local and global information at each step, and generating a more effective sub-model architecture.
6. A laryngeal electrical signal classification device based on neural architecture search, It is characterized in that include: A data acquisition module, used to acquire and preprocess laryngeal electrical signal data, and divide the preprocessed laryngeal electrical signal data into a training set, a validation set, and a test set; A sub-model building module, for generating a sub-model through a controller, wherein the controller is a model for executing a neural architecture search method; A sub-model optimization module, used to obtain a small batch data set from the training set to train the sub-model, and use the validation set to determine the classification accuracy of the trained sub-model; A controller optimization module, used for taking the classification accuracy as a reward of the controller, updating the controller by the reward, and regenerating a sub-model by the updated controller, and taking the sub-model with the highest classification accuracy as a laryngeal electrical signal classification model; The classification model determination module is used to train the laryngeal electrical signal classification model through the training set, and use the trained laryngeal electrical signal classification model to classify the laryngeal electrical signal data in the test set.
7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the laryngeal electrical signal classification method based on neural architecture search as described in any one of claims 1 to 5.
8. An electronic device, It is characterized in that It includes a processor and a memory, wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the laryngeal electrical signal classification method based on neural architecture search as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Electroencephalogram signal emotion recognition method based on network structure search
CN113516101A
Human motion intention recognition method based on network architecture search
CN113962268A
Mobile robot complex intersection anti-collision method and system
CN114454162A
Automatic construction of neural network architecture using bayesian graph exploration
CN115769228A
Cloud edge cooperative training method and system based on federated learning and neural architecture search
CN116070714A
