An ERNIE_CN-GRU step automatic recognition method, system, device and medium

Abstract

The application provides an ERNIE_CN-GRU sentence step automatic identification method, system, device and medium, complete data of a paragraph is acquired, and a data set is constructed; an ERNIE pre-training model is built, a multi-head self-attention mechanism is fused to learn text semantics to obtain a word vector feature matrix of the multi-head attention mechanism; CN-GRU feature identification network training is carried out based on the ERNIE pre-training model and the word vector feature matrix, and an ERNIE_CN-GRU model is formed; the data set is input into the built ERNIE_CN-GRU model, a Softmax classifier is accessed to realize sentence step identification, and an identification label is output; the ERNIE pre-training model combining large-scale text content and a knowledge graph is used to learn deep text semantics, the disadvantages that a traditional machine learning does not sufficiently mine and utilize the internal relationship and features between words are improved, and good transferability and robustness are obtained.

Description

Technical Field

[0001] This invention belongs to the field of text processing technology, specifically relating to an automatic speech step recognition method, system, device, and medium based on ERNIE_CN-GRU. Background Technology

[0002] Text classification is a classic and important task in natural language processing. It involves learning the semantics and features of text and identifying categories based on this information. For researchers, reading and quickly extracting the key points from articles is an effective way to improve research efficiency. Therefore, paragraph recognition in paper abstracts is particularly important. In academic paper abstract paragraph recognition, concise and clear paragraph words are used to summarize the abstract sentences, helping readers quickly locate specific information in the paper. Paragraph classification also has applications in areas such as artificial intelligence recommendation, library and information science, information mining, knowledge discovery, and knowledge graph construction. It is evident that paragraph recognition can serve as a fundamental task in text processing research and is of significant importance. However, existing paragraph recognition algorithms still need improvement because they do not fully explore and utilize the inherent relationships and features between words.

[0003] For text classification problems, researchers generally use methods based on traditional machine learning and deep learning. Traditional machine learning and deep learning methods for identifying speech steps in summaries are mostly based on structured summaries or English datasets. However, in rule-based methods, rule formulation is crucial for recognition performance, but rules cannot cover all speech step cases, so the recognition performance needs to be further improved. Speech step recognition methods based on traditional machine learning and deep learning rely on the lexical and morphological features of the text, but machine learning cannot learn its semantics in depth, so the performance is not optimal.

[0004] For text classification, natural language has the problem of diverse and complex expressions, which makes it difficult for machines to understand; moreover, there is currently no perfect mathematical model to accurately describe the content expressed by natural language, which poses a great challenge to machine semantic understanding of natural language. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an automatic speech step recognition method, system, device, and medium based on ERNIE_CN-GRU, which can improve the accuracy of speech step recognition.

[0006] This invention is achieved through the following technical solution:

[0007] An automatic speech gait recognition method based on ERNIE_CN-GRU includes the following steps:

[0008] S1: Obtain the complete data for the paragraph and build the dataset;

[0009] S2: Build an ERNIE pre-trained model, integrate multi-head self-attention mechanism to learn text semantics and obtain word vector feature matrix fused with multi-head attention mechanism;

[0010] S3: Train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model;

[0011] S4: Input the dataset into the constructed ERNIE_CN-GRU model, connect the Softmax classifier to realize speech gait recognition, and output the recognition label.

[0012] Furthermore, in step S1, after obtaining the complete data of the paragraph, the text format is standardized and format symbols such as "\n", "\t", and spaces are cleaned from the text, while retaining the complete text content of the original data; the obtained text data is tagged according to the determined word step labels and divided into training data and test data in an 8:2 ratio.

[0013] Furthermore, in step S2, when building the ERNIE pre-trained model, the Transformer-XL feature processor is used to fuse a multi-head self-attention mechanism to learn text semantics and obtain a word vector feature matrix fused with the multi-head attention mechanism.

[0014] Furthermore, the step S2 of building the ERNIE pre-trained model includes the following steps:

[0015] A1: Write the pre-trained model call interface to load the pre-training parameter information required by the pre-trained model; the ERNIE pre-trained model uses a three-segment single-word mask, phrase and entity-level mask information integration method to obtain the complete semantics of words, phrases and entities;

[0016] A2: Word vector X:{w after three-segment masking i1 ,w i2 ,...,w iN The input is a Transformer-XL encoder, which undergoes a word embedding process. it =W e w it ,t∈[1,N],W e The weight parameters of the Embedding layer are used to transform the high-dimensional sparse word vector matrix into a low-dimensional dense word vector matrix, which is the word embedding vector for each sentence.

[0017] A3: For the three weight matrices W obtained from a single self-attention calculation q W k W vThe matrices Q, K, and V obtained by multiplying the word embedding vector by the three factors respectively represent the relevance between the current word and other words in the sentence. To prevent the results from being too large, they are divided by the root mean square of their dimensions. d k Let represent the dimension of a Q or K matrix vector, where The learnable variable is used for relative distance calculation within a certain range. Then, it is normalized using the Softmax function to obtain the relevance of each word to other words after normalization. This is then multiplied by the V matrix, i.e., a weighted summation, to obtain a new vector code for each word. The formula is as follows:

[0018]

[0019] A4: Merge the Q, K, and V matrices calculated from each individual self-attention according to the segmented head, and multiply them by the weights W. 0 The segmented head is linearly transformed into a matrix of the original dimensions to obtain the Multi-Head matrix. This process can be represented as follows:

[0020] head i =Attention(Q) i K i V i ), i = 1, ..., h;

[0021] MultiHead(Q,K,V)=Concact(head i ,...,head h W 0 ;

[0022] A5: The Q, K, and V matrices obtained from the Multi-Head calculation are input into the Add&Norm layer for self-attention input-output summation and normalization. Then, after processing by the feedforward neural network of the fully connected layer and the Add&Norm layer, a word vector feature matrix with multi-head attention mechanism is obtained. The matrix contains the text features learned by the model and the contextual semantic information contained in the text.

[0023] Furthermore, the formation of the ERNIE_CN-GRU model in step S3 includes the following steps:

[0024] B1: Input the word vector feature matrix output by the pre-trained model into the convolutional layer of the CNN model, and extract the features of the deep semantic knowledge of the text learned by the pre-trained model through the convolutional layer;

[0025] B2: Input the text semantic features extracted from the convolutional layers of the CNN model into the pooling layers of the CNN model, and filter the text semantic features through the pooling layers;

[0026] B3: Input the filtered text semantic data into the fully connected FC layer of the CNN model to transform the non-linear text semantic data features into linear text semantic data features, which makes it easier to further build a prediction model through the linear text semantic data features;

[0027] B4: Input the linear text semantic data features output by the CNN into the BIGRU bidirectional gating network to further extract text semantic features.

[0028] Furthermore, the convolution operation and feature mapping process in the multi-layer convolution are as follows:

[0029]

[0030]

[0031] Where * represents a convolution operation. and These are the input and output of the convolution kernel, and f() is the activation function. This is the input to the activation function;

[0032] A basic convolution operation block is represented as:

[0033]

[0034] h = ReLU(s);

[0035] in, This represents a convolution operation. A basic convolution operation block contains a convolutional layer, a batch normalization layer, and the ReLU activation function.

[0036] In step B4, the BIGRU bidirectional gating network includes a reset gate and an update gate. The reset gate is used to filter out some state information from the previous time step from the candidate states; the update gate determines the amount of information from the historical states to be retained in the current state, as shown in the formula:

[0037] r t =(W r x t + r h t-1 );

[0038] Where δ is the activation function, x t h is the current input. t-1 The output of the hidden layer at the previous time step is the historical state, W. r and U r This is the weight matrix;

[0039] The bidirectional gated network trains GRUs in both forward and backward directions to obtain textual contextual features. The forward operation involves training the forward hidden state of the i-th sentence. and backward hidden state for:

[0040]

[0041]

[0042] The sentence encoding representation is obtained through the implicit states before and after:

[0043]

[0044] The forward and backward training connections are used to update the current gate state, which is achieved by creating a new candidate value u from the tanh layer. i Decision:

[0045] u i =tanh(W s h i +b s )

[0046] By applying the linear text semantic data features obtained from the CNN model to the bidirectional gating mechanism BIGRU, feature vectors in the text can be further extracted, allowing for a deeper exploration of the contextual features and relevance of the text data.

[0047] Furthermore, step S4 includes the following steps:

[0048] The processed data is input into the model, the pre-trained model interface is called to perform text segmentation, and the segmented words are vectorized according to the dictionary mapping of the pre-trained model.

[0049] C1: Based on the batch value, the length of each sentence in the training and test data is pad_size. The text is masked with 0 / 1. The text word vector content, label numbers and mask results are merged and stored in a .pkl file. When loading, it is loaded into a DataFrame table data structure using a tabular data structure.

[0050] C2: Input the vectorized DataFrame structured data into the ERNIE_CN-GRU model in batches, and add a Dropout layer after the model to randomly ignore a preset number of neurons to prevent overfitting.

[0051] C3: Connects to the fully connected layer, uses Softmax for speech gait recognition, and outputs classification labels;

[0052] C4: At the start of training, after each round of forward computation and backpropagation, label prediction is performed on the test data. During backpropagation, cross-entropy is used as the loss function to optimize the model.

[0053]

[0054] Where D is the size of the training data and C is the number of classes. For text data labels, To predict the probability of the model, the optimal prediction result label is determined by multiple parameter tunings. This yields the speech step recognition effect and well-classified test label data, namely the speech step classification result and the loss change analysis model during model operation.

[0055] An automatic speech gait recognition system based on ERNIE_CN-GRU includes:

[0056] The data acquisition module is used to obtain complete data from paragraphs and build datasets.

[0057] The model building module is used to build the ERNIE pre-trained model and learn text semantics by fusing a multi-head self-attention mechanism to obtain a word vector feature matrix that fuses the multi-head attention mechanism;

[0058] The training model module is used to train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model.

[0059] The output module is used to input the dataset into the constructed ERNIE_CN-GRU model, connect it to the Softmax classifier to realize gait recognition, and output the recognition label.

[0060] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the ERNIE_CN-GRU automatic speech recognition method.

[0061] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the ERNIE_CN-GRU-based automatic speech recognition method.

[0062] Compared with the prior art, the present invention has the following beneficial technical effects:

[0063] This invention provides an automatic speech step recognition method, system, device, and medium based on ERNIE_CN-GRU. It involves acquiring complete paragraph data and constructing a dataset; building an ERNIE pre-trained model and integrating a multi-head self-attention mechanism to learn text semantics, resulting in a word vector feature matrix fused with the multi-head attention mechanism; training a CN-GRU feature recognition network based on the ERNIE pre-trained model and the word vector feature matrix to form an ERNIE_CN-GRU model; inputting the dataset into the constructed ERNIE_CN-GRU model and connecting it to a Softmax classifier to achieve speech step recognition and output recognition labels. This invention utilizes ERN, which combines large-scale text content with knowledge graphs. The IE pre-trained model learns deep semantics from text, improving upon the shortcomings of traditional machine learning in not fully exploring and utilizing the inherent relationships and features between words. Downstream convolutional neural networks and bidirectional gated network CN-GRU models perform convolutional feature extraction. The convolutional operations give the model local connectivity and weight sharing characteristics, thus facilitating the extraction of overall text features by integrating local information. Furthermore, feature dimensionality reduction in the pooling layer effectively mitigates overfitting, reduces algorithmic overhead, and obtains valuable information. The bidirectional GRU network model can quickly learn information from previous and future time points, exhibiting stronger memory for long sentences, thus compensating for the poor classification performance caused by the forgetting problem in machine learning due to long text inputs. In addition, this invention generates a word vector matrix through dictionary mapping of the pre-trained model, using machine-readable numerical matrices to represent text. Through deep semantic learning by the pre-trained model, good results can be achieved without a large number of test data samples. Moreover, the pre-trained model learns from large-scale text data and knowledge graph knowledge, exhibiting good transferability. Combined with its own training data for semantic learning training, it is more robust to test data. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the implementation of the overall structure of the automatic speech recognition based on ERNIE_CN-GRU in this invention.

[0065] Figure 2 This is a diagram of the automatic speech recognition model based on ERNIE_CN-GRU in this invention;

[0066] Figure 3 This is a schematic diagram of the ERNIE pre-trained model;

[0067] Figure 4 Example diagram of the MASK method for ERNIE pre-trained models;

[0068] Figure 5 This is a diagram of the CN-GRU neural network structure in this invention;

[0069] Figure 6 This describes the computation process of the CNN layer in this invention. Detailed Implementation

[0070] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0071] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0072] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0073] This invention provides an automatic speech gait recognition method based on ERNIE_CN-GRU, such as... Figure 1 As shown, it includes the following steps:

[0074] S1: Obtain the complete data for the paragraph and build the dataset;

[0075] S2: Build an ERNIE pre-trained model, integrate multi-head self-attention mechanism to learn text semantics and obtain word vector feature matrix fused with multi-head attention mechanism;

[0076] S3: Train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model;

[0077] S4: Input the dataset into the constructed ERNIE_CN-GRU model, connect the Softmax classifier to realize speech gait recognition, and output the recognition label.

[0078] Preferred, such as Figure 2As shown, in step S1, after obtaining the complete data of the paragraph, the text format is standardized and format symbols such as "\n", "\t", and spaces are cleaned from the text, while retaining the complete text content of the original data; the obtained text data is tagged according to the determined word step labels and divided into training data and test data in an 8:2 ratio.

[0079] Preferred, such as Figure 3 As shown, in step S2, when building the ERNIE pre-trained model, the Transformer-XL feature processor is used to fuse a multi-head self-attention mechanism to learn text semantics and obtain a word vector feature matrix fused with the multi-head attention mechanism.

[0080] Preferably, the step S2 of building the ERNIE pre-trained model includes the following steps:

[0081] A1: Write the pre-trained model call interface to load the pre-training parameter information required by the pre-trained model; the ERNIE pre-trained model uses a three-segment single-word mask, phrase, and entity-level mask information integration method to obtain the complete semantics of words, phrases, and entities, such as... Figure 4 As shown, for the input text sentence, firstly, words, Chinese characters, and other information in the sentence are treated as basic language units and randomly masked by 15%, resulting in a basic-level mask to obtain the basic-level information of the sentence. Next, a second-stage entity-level mask is performed, which masks proper nouns and other entities in the sentence and predicts the gaps in the entities. Finally, a phrase-specific mask is performed, which masks the phrases in the sentence and predicts all basic units in the same phrase. The phrase information is encoded into the word embedding. By using three segments of masks, the text information of different semantic units is obtained, resulting in a rich expression of the semantic information of the sentence.

[0082] A2: Word vector X:{w after three-segment masking i1 ,w i2 ,...,w iN The input is a Transformer-XL encoder, which undergoes a word embedding process. it =W e w it ,t∈[1,N],W e The weight parameters of the Embedding layer are used to transform the high-dimensional sparse word vector matrix into a low-dimensional dense word vector matrix, which is the word embedding vector for each sentence.

[0083] A3: For the three weight matrices W obtained from a single self-attention calculation q W k W vThe matrices Q, K, and V obtained by multiplying the word embedding vector by the three factors respectively represent the relevance between the current word and other words in the sentence. To prevent the results from being too large, they are divided by the root mean square of their dimensions. d k Let represent the dimension of a Q or K matrix vector, where The learnable variable is used for relative distance calculation within a certain range. Then, it is normalized using the Softmax function to obtain the relevance of each word to other words after normalization. This is then multiplied by the V matrix, i.e., a weighted summation, to obtain a new vector code for each word. The formula is as follows:

[0084]

[0085] A4: Merge the Q, K, and V matrices calculated from each individual self-attention according to the segmented head, and multiply them by the weights W. 0 The segmented head is linearly transformed into a matrix of the original dimensions to obtain the Multi-Head matrix. This process can be represented as follows:

[0086] head i =Attention(Q) i K i V i ), i = 1, ..., h;

[0087] MultiHead(Q,K,V)=Concact(head i ,...,head h ) 0 ;

[0088] A5: The Q, K, and V matrices obtained from the Multi-Head calculation are input into the Add&Norm layer for self-attention input-output summation and normalization. Then, after processing by the feedforward neural network of the fully connected layer and the Add&Norm layer, a word vector feature matrix with multi-head attention mechanism is obtained. The matrix contains the text features learned by the model and the contextual semantic information contained in the text.

[0089] Preferred, such as Figure 5 As shown, the formation of the ERNIE_CN-GRU model in step S3 includes the following steps:

[0090] B1: Input the word vector feature matrix output by the pre-trained model into the convolutional layer of the CNN model, and extract the features of the deep semantic knowledge of the text learned by the pre-trained model through the convolutional layer;

[0091] B2: Input the text semantic features extracted from the convolutional layers of the CNN model into the pooling layers of the CNN model, and filter the text semantic features through the pooling layers;

[0092] B3: Input the filtered text semantic data into the fully connected FC layer of the CNN model to transform the non-linear text semantic data features into linear text semantic data features, which makes it easier to further build a prediction model through the linear text semantic data features;

[0093] B4: Input the linear text semantic data features output by the CNN into the BIGRU bidirectional gating network to further extract text semantic features.

[0094] Furthermore, such as Figure 6 As shown, the convolution operation and feature mapping process in the multi-layer convolution is as follows:

[0095]

[0096]

[0097] Where * represents a convolution operation. and These are the input and output of the convolution kernel, and f() is the activation function. This is the input to the activation function;

[0098] A basic convolution operation block is represented as:

[0099]

[0100] h = ReLU(s);

[0101] in, This represents a convolution operation. A basic convolution operation block contains a convolutional layer, a batch normalization layer, and the ReLU activation function.

[0102] In step B4, the BIGRU bidirectional gating network includes a reset gate and an update gate. The reset gate is used to filter out some state information from the previous time step from the candidate states; the update gate determines the amount of information from the historical states to be retained in the current state, as shown in the formula:

[0103] r t =δ(W r x t +U r h t-1 );

[0104] Where δ is the activation function, x t h is the current input. t-1 The output of the hidden layer at the previous time step is the historical state, W. r and U r This is the weight matrix;

[0105] The bidirectional gated network trains GRUs in both forward and backward directions to obtain textual contextual features. The forward operation involves training the forward hidden state of the i-th sentence. and backward hidden state for:

[0106]

[0107]

[0108] The sentence encoding representation is obtained through the implicit states before and after:

[0109]

[0110] The forward and backward training connections are used to update the current gate state, which is achieved by creating a new candidate value u from the tanh layer. i Decision:

[0111] u i =anh(W s h i + s )

[0112] By applying the linear text semantic data features obtained from the CNN model to the bidirectional gating mechanism BIGRU, feature vectors in the text can be further extracted, allowing for a deeper exploration of the contextual features and relevance of the text data.

[0113] Furthermore, step S4 includes the following steps:

[0114] The processed data is input into the model, the pre-trained model interface is called to perform text segmentation, and the segmented words are vectorized according to the dictionary mapping of the pre-trained model.

[0115] C1: Based on the batch value, the length of each sentence in the training and test data is pad_size. The text is masked with 0 / 1. The text word vector content, label numbers and mask results are merged and stored in a .pkl file. When loading, it is loaded into a DataFrame table data structure using a tabular data structure.

[0116] C2: Input the vectorized DataFrame structured data into the ERNIE_CN-GRU model in batches, add a Dropout layer after the model, and randomly ignore a preset number of neurons to prevent the model from overfitting.

[0117] C3: Connects to the fully connected layer, uses Softmax for speech gait recognition, and outputs classification labels;

[0118] C4: At the start of training, after each round of forward computation and backpropagation, label prediction is performed on the test data. During backpropagation, cross-entropy is used as the loss function to optimize the model.

[0119]

[0120] Where D is the size of the training data and C is the number of classes. For text data labels, To predict the probability of the model, the optimal prediction result label is determined by multiple parameter tunings. This yields the speech step recognition effect and well-classified test label data, namely the speech step classification result and the loss change analysis model during model operation.

[0121] This invention provides an automatic speech gait recognition system based on ERNIE_CN-GRU, comprising:

[0122] The data acquisition module is used to obtain complete data from paragraphs and build datasets.

[0123] The model building module is used to build the ERNIE pre-trained model and learn text semantics by fusing a multi-head self-attention mechanism to obtain a word vector feature matrix that fuses the multi-head attention mechanism;

[0124] The training model module is used to train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model.

[0125] The output module is used to input the dataset into the constructed ERNIE_CN-GRU model, connect it to the Softmax classifier to realize gait recognition, and output the recognition label.

[0126] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for operation based on an ERNIE_CN-GRU automatic speech recognition method.

[0127] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the ERNIE_CN-GRU-based automatic speech recognition method described in the above embodiments.

[0128] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic speech gait recognition method based on ERNIE_CN-GRU, characterized in that, Includes the following steps: S1: Obtain the complete data for the paragraph and build the dataset; S2: Build an ERNIE pre-trained model, integrate multi-head self-attention mechanism to learn text semantics and obtain word vector feature matrix fused with multi-head attention mechanism; When building the ERNIE pre-trained model, the Transformer-XL feature processor is used to fuse a multi-head self-attention mechanism to learn text semantics and obtain a word vector feature matrix fused with the multi-head attention mechanism; Building an ERNIE pre-trained model involves the following steps: A1: Write the pre-trained model call interface to load the pre-training parameter information required by the pre-trained model; the ERNIE pre-trained model uses a three-segment single-word mask, phrase and entity-level mask information integration method to obtain the complete semantics of words, phrases and entities; A2: Word vector after three-segment masking The input is to the Transformer-XL encoder, and then undergoes the word embedding process. , The weight parameters of the Embedding layer are used to transform the high-dimensional sparse word vector matrix into a low-dimensional dense word vector matrix, which is the word embedding vector for each sentence. A3: For the three weight matrices calculated for a single self-attention... , , The matrices Q, K, and V obtained by multiplying the word embedding vector by the three factors respectively represent the relevance between the current word and other words in the sentence. To prevent the results from being too large, they are divided by the root mean square of their dimensions. , Represents a Q or The dimension of a matrix or vector, where The learnable variable is used for relative distance calculation within a certain range. Then, it is normalized using the Softmax function to obtain the relevance of each word to other words after normalization. This is then multiplied by the V matrix, i.e., a weighted summation, to obtain a new vector code for each word. The formula is as follows: ； A4: Combine the Q, K, and V matrices calculated from each individual self-attention according to the segmented head, and multiply the weights by the dot product. The segmented head is linearly transformed into a matrix of the original dimensions to obtain the Multi-Head matrix. This process can be represented as follows: ； ； A5: Input the Q, K, and V matrices obtained from the Multi-Head calculation into the Add&Norm layer for self-attention input-output summation and normalization. Then, process them through the feedforward neural network of the fully connected layer and the Add&Norm layer to obtain the word vector feature matrix that integrates the multi-head attention mechanism. The matrix contains the text features learned by the model and the contextual semantic information contained in the text. S3: Train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model; The steps involved in forming the ERNIE_CN-GRU model are as follows: B1: Input the word vector feature matrix output by the pre-trained model into the convolutional layer of the CNN model, and extract the features of the deep semantic knowledge of the text learned by the pre-trained model through the convolutional layer; B2: Input the text semantic features extracted from the convolutional layers of the CNN model into the pooling layers of the CNN model, and filter the text semantic features through the pooling layers; B3: Input the filtered text semantic data into the fully connected FC layer of the CNN model to transform the non-linear text semantic data features into linear text semantic data features, which makes it easier to further build a prediction model through the linear text semantic data features; B4: Input the linear text semantic data features output by the CNN into the BIGRU bidirectional gating network to further extract text semantic features; S4: Input the dataset into the constructed ERNIE_CN-GRU model, connect the Softmax classifier to realize speech gait recognition, and output the recognition label.

2. The method for automatic speech gait recognition based on ERNIE_CN-GRU according to claim 1, characterized in that, In step S1, after obtaining the complete data of the paragraph, the text format is standardized and the format symbols "\n", "\t", and spaces in the text are cleaned, while retaining the complete text content of the original data. The obtained text data is tagged according to the determined word step labels and divided into training data and test data in an 8:2 ratio.

3. The automatic speech gait recognition method based on ERNIE_CN-GRU according to claim 1, characterized in that, The convolution operation and feature mapping process in the convolutional layer are as follows: Where * represents a convolution operation. These are the input and output of the convolution kernel. For activation function, This is the input to the activation function; A basic convolution operation block is represented as: ； ； in, This represents a convolution operation. A basic convolution operation block contains a convolutional layer, a batch normalization layer, and the ReLU activation function. In step B4, the BIGRU bidirectional gating network includes a reset gate and an update gate. The reset gate is used to filter out some state information from the previous time step from the candidate states; the update gate determines the amount of information from the historical states to be retained in the current state, as shown in the formula: ； in, For activation function, For the current input, This is the output of the hidden layer from the previous moment, which represents the historical state. and This is the weight matrix; The bidirectional gated network trains GRUs in both forward and backward directions to obtain textual contextual features. The forward operation involves training the forward hidden state of the i-th sentence. and backward hidden state for: ； ； The sentence encoding representation is obtained through the implicit states before and after: ； The forward and backward training connections are used to update the current gate state, which is a new candidate value created by the tanh layer. Decision: By applying the linear text semantic data features obtained from the CNN model to the bidirectional gating mechanism BIGRU, feature vectors in the text can be further extracted, allowing for a deeper exploration of the contextual features and relevance of the text data.

4. The automatic speech gait recognition method based on ERNIE_CN-GRU according to claim 1, characterized in that, Step S4 includes the following steps: The processed data is input into the model, the pre-trained model interface is called to perform text segmentation, and the segmented words are vectorized according to the dictionary mapping of the pre-trained model. C1: Based on the batch value, the length of each sentence in the training and test data is pad_size. The text is 0 / 1 masked, and the text word vector content, label number and mask result are merged and stored as a .pkl file. When loading, it is loaded as a DataFrame table data structure using a tabular data structure method. C2: Input the vectorized DataFrame structured data into the ERNIE_CN-GRU model in batches, add a Dropout layer after the model, and randomly ignore a preset number of neurons to prevent the model from overfitting. C3: Connects to the fully connected layer, uses Softmax for speech gait recognition, and outputs classification labels; C4: At the start of training, after each round of forward computation and backpropagation, label prediction is performed on the test data. During backpropagation, cross-entropy is used as the loss function to optimize the model. ； Where D is the size of the training data and C is the number of classes. For text data labels, To predict the probability of the model, the optimal prediction result label is determined by multiple parameter tunings. This yields the speech step recognition effect and well-classified test label data, namely the speech step classification result and the loss change analysis model during model operation.

5. An automatic speech gait recognition system based on ERNIE_CN-GRU, characterized in that, The method for automatic speech recognition based on ERNIE_CN-GRU according to any one of claims 1-4 includes: The data acquisition module is used to obtain complete data from paragraphs and build datasets. The model building module is used to build the ERNIE pre-trained model and learn text semantics by fusing a multi-head self-attention mechanism to obtain a word vector feature matrix that fuses the multi-head attention mechanism; The training model module is used to train the CN-GRU feature recognition network based on the ERNIE pre-trained model and word vector feature matrix to form the ERNIE_CN-GRU model. The output module is used to input the dataset into the constructed ERNIE_CN-GRU model, connect it to the Softmax classifier to realize gait recognition, and output the recognition label.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the automatic speech recognition method based on ERNIE_CN-GRU as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the automatic speech recognition method based on ERNIE_CN-GRU as described in any one of claims 1-4.

Images