Voice keyword recognition method, system, device and storage medium

Abstract

The present invention provides a method, system, device and storage medium for speech keyword recognition, the method comprising: a training part: establishing a training database, including sample audios of multiple keywords; dividing each sample audio into multiple sample audio segments; extracting feature vectors of each sample audio segment, making the feature vectors into a feature matrix for input training algorithm according to the input data requirements of image classification training algorithm, and performing model training, and storing the trained model; a recognition part: collecting audio to be recognized, dividing it into multiple audio segments to be recognized, extracting feature vectors of each audio segment to be recognized, making it into a feature matrix suitable for the model, inputting the model, and obtaining labels of each audio segment to be recognized; using the label with the largest number as the keyword label of the audio to be recognized. The present invention realizes keyword recognition based on machine learning image classification. It has the advantages of less training samples, short training time, low algorithm complexity, short recognition reasoning time, and high recognition rate.

Description

Background Art

[0002] Voice keyword recognition is the core technology of intelligent voice system. It can be applied to real-time dialogue system, command control function, voice dialing function, voice monitoring function, etc. in intelligent voice system. It is an important part of intelligent control of voice equipment. Generally speaking, when applied in intelligent voice system, the keywords are fixed, with low memory usage, low computational complexity, low power consumption and high accuracy.

[0003] Voice keyword recognition can realize real-time retrieval, review and effective monitoring of audio information on the Internet, such as online TV, audio and video, and copyright protection of various digital audio products, such as music, that is, searching for unauthorized use.

[0004] Speech keyword recognition also plays an important role in the research of audio information classification and statistical technology. For example, in radio and television news programs, digital libraries and other contents, there are a lot of speech, music and other information. The use of audio information retrieval technology can effectively classify, count and retrieve this information, better manage, store and efficiently use these resources. In addition, it can also be applied to many fields such as network management, information security and military criminal investigation.

[0005] When voice keyword recognition is applied to the classification and statistics of media or audio information on the Internet, the number of keywords may be large. If machine learning models and deep learning models are used for recognition, a large number of training samples are required for training, and the memory usage rate is high, the calculation complexity is high, and the power consumption is very good. At the same time, due to the diversity of audio content, the accuracy of voice keyword recognition is difficult to guarantee. Summary of the invention

[0006] In view of the problems in the prior art, the purpose of the present invention is to provide a method, system, device and storage medium for voice keyword recognition, which reduces the algorithm complexity of voice keyword recognition and improves the accuracy of voice keyword recognition.

[0007] The embodiment of the present invention provides a method for recognizing speech keywords, including a training part and a recognition part, wherein the training part includes the following steps:

[0008] Establishing a training database, wherein the training database includes sample audios of a plurality of keywords;

[0009] Divide each sample audio into a plurality of sample audio segments;

[0010] Extracting feature vectors of each sample audio segment, and making the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm;

[0011] Performing model training based on the feature matrix of the input training algorithm, and storing the trained keyword recognition model;

[0012] The identification part includes the following steps:

[0013] Collecting audio to be recognized, dividing it into multiple audio segments to be recognized, and extracting feature vectors of each audio segment to be recognized;

[0014] The feature vector to be identified is made into a feature matrix suitable for inputting into the keyword recognition model, and the trained keyword recognition model is inputted to obtain the label of each audio segment to be identified;

[0015] Count the labels of each audio segment to be identified in the audio to be identified, and use the label with the largest number as the keyword label of the audio to be identified.

[0016] The speech keyword recognition method of the present invention first collects sample audio during the training process, then divides the sample audio into multiple sample audio segments, then extracts the sample audio segments and processes them into feature matrices that meet the requirements of the training algorithm to perform model training, and applies the algorithm commonly used for image classification to speech, and retains the audio data, constructs the data format, and then directly inputs the algorithm. The keyword recognition model can be trained with a small number of samples, the training time is short, and good recognition results can be achieved based on training with a small amount of data.

[0017] During the recognition process, the audio to be recognized is first collected and divided into multiple audio segments to be recognized, and then features are extracted for each audio segment to be recognized, and the feature vectors of each audio segment to be recognized are combined to obtain a feature matrix. The keyword recognition model is used to obtain the keyword label of each audio segment to be recognized, and the keyword labels of all audio segments to be recognized are counted, and the keyword label with the largest number is used as the keyword label of the entire audio to be recognized. The recognition method is simple, which greatly improves the accuracy of speech keyword recognition, has low algorithm complexity, and short recognition reasoning time.

[0018] In some embodiments, the training algorithm is XGBoost.

[0019] In some embodiments, dividing each sample audio into a plurality of sample audio segments includes dividing each sample audio into at least one of a short-time audio segment, a medium-time audio segment, and a long-time audio segment;

[0020] Dividing the audio to be recognized into a plurality of audio segments to be recognized, including dividing each audio to be recognized into at least one of a short-term audio segment, a medium-term audio segment, and a long-term audio segment;

[0021] The short-term audio segment, the medium-term audio segment, and the long-term audio segment are respectively an audio segment including a1 frames, an audio segment including a2 frames, and an audio segment including a3 frames. <a2<a3。

[0022] In some embodiments, the step of extracting feature vectors of each audio segment to be identified comprises the following steps:

[0023] Determine the category of the required feature vector;

[0024] If a short-time feature vector is needed, the short-time feature vector of the audio segment to be recognized is extracted based on a preset speech feature extraction algorithm;

[0025] If a medium-time or long-time feature vector is needed, the feature vectors of each short-time audio segment contained in the audio segment to be identified are obtained, and the medium-time or long-time feature vector of the audio segment to be identified is obtained based on a preset feature statistical algorithm.

[0026] In some embodiments, the feature matrix data is made into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm, including the following steps:

[0027] The feature vectors of each sample audio segment of all sample audios are extracted, and the keyword label of the sample audio segment is added to the last column or the last row of the feature vector, and the feature vectors of each sample audio segment of all sample audios are combined to obtain a feature matrix.

[0028] In some embodiments, combining the feature vectors of each sample audio segment of all sample audios to obtain a feature matrix includes the following steps:

[0029] The feature vector of each sample audio segment is taken as a row in the feature matrix;

[0030] Using a random number seed generator, the order of rows in the feature matrix where all eigenvectors are located is disrupted to obtain the updated row order of all eigenvectors;

[0031] Based on the updated row order of all eigenvectors, all eigenvectors are then combined to get the final feature matrix, which is input into the XGBoost training algorithm.

[0032] In some embodiments, after establishing the training database, the following steps are also included:

[0033] Determine whether the duration of the sample audio is less than a preset duration threshold;

[0034] If so, the sample audio is repeated a specific number of times to obtain a new sample audio, and the new sample audio is used to replace the sample audio before being repeated the specific number of times.

[0035] The embodiment of the present invention further provides a speech keyword recognition system, which is applied to the speech keyword recognition method, and the system includes:

[0036] An audio segmentation module is used to establish a training database, wherein the training database includes sample audios of multiple keywords, divide each sample audio into multiple sample audio segments, and collect audio to be recognized and divide it into multiple audio segments to be recognized;

[0037] A feature extraction module is used to extract feature vectors of each sample audio segment, and make the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm; and extract feature vectors of each audio segment to be identified, and make the feature vectors to be identified into a feature matrix suitable for input into the keyword recognition model;

[0038] A model training module, used for performing model training based on the feature matrix of the input training algorithm and storing the trained keyword recognition model;

[0039] The audio recognition module is used to input the feature matrix to be recognized into the trained keyword recognition model to obtain the label of each audio segment to be recognized, and to count the labels of each audio segment to be recognized in the audio to be recognized, and to use the label with the largest number as the keyword label of the audio to be recognized.

[0040] The speech keyword recognition system of the present invention, during the training process, first collects sample audio through the audio segmentation module, divides the sample audio into multiple sample audio segments, then extracts the sample audio segments through the feature extraction module and processes them into feature matrices that meet the requirements of the training algorithm, and performs model training through the model training module. In the feature extraction module and the model training module, the algorithm commonly used for image classification is applied to speech, and the audio data is retained, the data format is constructed, and then directly input into the algorithm. The keyword recognition model can be trained with a small amount of samples, and the training time is short. Good recognition results can also be achieved based on training with a small amount of data. The present invention has made an attempt to use the algorithm commonly used for image processing for speech processing, and it does not generate a spectrogram from the speech, and then use it as an image, and then use the image algorithm. Instead, it directly extracts data that effectively represents the speech, such as speech feature values, from the speech audio file, and uses the algorithm for training, recognition and labeling. The algorithm results show that the effect is very good.

[0041] During the recognition process, the audio to be recognized is first collected by using an audio segmentation module and divided into multiple audio segments to be recognized, and then the feature extraction module is used to extract features from each audio segment to be recognized, and the feature vectors of each audio segment to be recognized are combined to obtain a feature matrix. The keyword recognition model is used by the audio recognition module to obtain the keyword label of each audio segment to be recognized, and the keyword labels of all audio segments to be recognized are counted, and the keyword label with the largest number is used as the keyword label of the entire audio to be recognized. The recognition method is simple, which greatly improves the accuracy of speech keyword recognition, has low algorithm complexity, and short recognition reasoning time.

[0042] The embodiment of the present invention further provides a speech keyword recognition device, which is characterized by comprising:

[0043] processor;

[0044] a memory storing executable instructions of the processor;

[0045] The processor is configured to execute the steps of the voice keyword recognition method by executing the executable instructions.

[0046] By adopting the voice keyword recognition device provided by the present invention, the processor executes the voice keyword recognition method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned voice keyword recognition method.

[0047] The embodiment of the present invention further provides a computer-readable storage medium for storing a program, wherein the program implements the steps of the voice keyword recognition method when executed by a processor.

[0048] By adopting the computer-readable storage medium provided by the present invention, the program stored therein implements the steps of the voice keyword recognition method when being executed, thereby obtaining the beneficial effects of the above-mentioned voice keyword recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0050] Figure 1 is a flow chart of the training part of the speech keyword recognition method according to an embodiment of the present invention;

[0051] Figure 2 is a flow chart of the recognition part of the speech keyword recognition method according to one embodiment of the present invention;

[0052] Figure 3is a flow chart of combining the feature vectors of all the sample audio segments of the sample audio to obtain a feature matrix according to an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of a speech keyword recognition system according to an embodiment of the present invention;

[0054] Figure 5 is a structural diagram of a speech keyword recognition device according to an embodiment of the present invention;

[0055] Figure 6 It is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.

[0057] like Figure 1 As shown, in one embodiment, the present invention provides a method for speech keyword recognition, including a training part and a recognition part, wherein the training part includes the following steps:

[0058] S110: Establishing a training database, wherein the training database includes sample audios of a plurality of keywords;

[0059] The process of establishing a training database is to collect sample audio for each keyword. The sample audio can be in wav format or other audio formats. The sample audio corresponding to the same keyword is placed in the same folder. The name of the folder can be the name of the corresponding keyword or the number of the corresponding keyword, that is, corresponding to the keyword label. Generally, at least 40 to 50 sample audios are collected for each keyword, that is, a small data volume voice sample library;

[0060] S120: Divide each sample audio into a plurality of sample audio segments;

[0061] Here, the sample audio segments are divided, for example, by treating each frame of the sample audio as an audio segment, or by setting every n frames of the sample audio as an audio segment, where n>1, and using the keyword tag of the sample audio as the keyword tag of each sample audio segment included;

[0062] S130: extracting feature vectors of each sample audio segment, and making the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm;

[0063] In some embodiments, the speech feature extraction algorithm pyAudioAnalysis can be used to extract features of each sample audio segment. pyAudioAnalysis is a very easy-to-use and powerful open source audio analysis tool that can realize feature extraction of audio (including music), training and execution of classification and regression models, as well as some other practical functions, such as audio segmentation and other processing.

[0064] In other alternative implementations, any effective feature extraction algorithm may be used to extract the features of the sample audio segment, and the feature extraction algorithm may be written by the user or by a mature code, such as a feature extraction algorithm written in Python code: librosa (a very powerful third-party library for Python speech signal processing), Yaffe (Yaafe coreaudio features, audio basic feature extraction), etc.

[0065] S140: Performing model training based on the feature matrix of the input training algorithm, and storing the trained keyword recognition model;

[0066] Storing the trained keyword recognition model here includes naming and saving the trained keyword recognition model;

[0067] The identification part includes the following steps:

[0068] S210: Collecting audio to be recognized;

[0069] The audio to be recognized may be in wav format or other audio formats. The audio to be recognized may be divided into segments, for example, each frame of the audio to be recognized may be regarded as an audio segment, or every n frames of the audio to be recognized may be regarded as an audio segment, where n>1;

[0070] S220: Divide the audio to be recognized into a plurality of audio segments to be recognized, and extract a feature vector of each audio segment to be recognized;

[0071] In some embodiments, the speech feature extraction algorithm pyAudioAnalysis can be used to extract features of each audio segment to be identified. pyAudioAnalysis is a very easy-to-use and powerful open source audio analysis tool that can realize feature extraction of audio (including music), training and execution of classification and regression models, as well as some other practical functions, such as audio segmentation and other processing.

[0072] In other alternative implementations, any effective feature extraction algorithm may be used to extract the features of the audio segment to be identified. The feature extraction algorithm may be written by the user or a mature code may be used. For example, a feature extraction algorithm written in Python code: librosa (a very powerful third-party library for Python speech signal processing), Yaffe (Yaafe coreaudio features, audio basic feature extraction), etc.

[0073] S230: The feature vector to be identified is made into a feature matrix suitable for inputting into the keyword recognition model, and the trained keyword recognition model is inputted to obtain labels of each audio segment to be identified;

[0074] S240: Count the labels of each audio segment to be recognized in the audio to be recognized, and use the label with the largest number as the keyword label of the audio to be recognized.

[0075] The speech keyword recognition method of the present invention, during the training process, first collects sample audio through step S110, then divides the sample audio into multiple sample audio segments through step S120, then extracts the sample audio segments through step S130 and processes them into feature matrices that meet the requirements of the training algorithm, and performs model training through step S140. In steps S130 and S140, the algorithm commonly used for image classification is applied to speech, and the audio data is retained, the data format is constructed, and then directly input into the algorithm. The keyword recognition model can be trained with a small number of samples, the training time is short, and good recognition results can be achieved based on training with a small amount of data. The present invention has made an attempt to use the algorithm commonly used for image processing for speech processing, and it does not generate a spectrogram from the speech, and then use it as an image, and then use the image algorithm. Instead, it directly extracts data that effectively represents the speech, such as speech feature values, from the speech audio file, and uses the algorithm for training, recognition and labeling. The algorithm results show that the effect is very good.

[0076] During the recognition process, firstly, the audio to be recognized is collected and divided into a plurality of audio segments to be recognized by adopting steps S210 and S220, and then, feature extraction is performed on each audio segment to be recognized by step S220, and the feature vectors of each audio segment to be recognized are combined to obtain a feature matrix by step S230, and a keyword recognition model is used to obtain a keyword label of each audio segment to be recognized, and the keyword labels of all audio segments to be recognized are counted by step S240, and the keyword label with the largest number is used as the keyword label of the entire audio to be recognized. The recognition method is simple, which greatly improves the accuracy of speech keyword recognition, has low algorithm complexity, and short recognition reasoning time.

[0077] In this embodiment, the image classification training algorithm adopted by the keyword recognition model is the XGBoost algorithm. XGBoost (eXtreme Gradient Boosting) combines hundreds or thousands of tree models into a highly accurate model. This model generates new trees through continuous iteration. XGBoost pre-sorts the data and then saves it as a block structure, which is repeatedly used in the training iteration, greatly reducing the computational amount. It is characterized by fast calculation speed and good model performance, and can be used in classification and regression problems. It is used in supervised learning, which means using training data xi with multiple features to predict the target variable yi. In the training data, yi is given as a label.

[0078] In this embodiment, the feature extraction of the audio segment adopts an improved pyAudioAnalysis algorithm. Specifically, the pyAudioAnalysis algorithm is mainly used, and multiple short-term, medium-term, and / or long-term feature extraction algorithms are added as supplements to extract effective feature values.

[0079] Specifically, in step S120: dividing each sample audio into multiple sample audio segments, including dividing the sample audio into at least one of short-term audio segments, medium-term audio segments, and long-term audio segments. In step S220: dividing the audio to be recognized into multiple audio segments to be recognized, including dividing the audio to be recognized into at least one of short-term audio segments, medium-term audio segments, and long-term audio segments. Specifically, which type of audio segment the sample audio or the audio to be recognized is divided into can be considered according to actual needs. For example, when applying this speech keyword recognition method to some instruction recognition scenarios, since the audio to be recognized is relatively short as a whole, it can only be divided into short-term audio segments. When applying this speech keyword recognition method to relatively long audio recognition, it can be divided into short-term audio segments and medium-term audio segments, or divided into short-term audio segments and long-term audio segments.

[0080] Among them, the short-term audio segment, the medium-term audio segment, and the long-term audio segment are respectively an audio segment including a1 frames, an audio segment including a2 frames, and an audio segment including a3 frames, where a1 < a2 < a3, and usually a1 = 1. For example, when a1 = 1, that is, each frame of audio is used as a short-term audio segment, the medium-term audio segment can include 20 - 50 frames, and the long-term audio segment can include 50 - 100 frames, etc. Specifically, the quantities of a1, a2, and a3 can be set according to needs.

[0081] Specifically, in this embodiment, in step S200: extracting the feature vectors of each audio segment to be recognized, including the following steps:

[0082] Judge the category of the required feature vectors;

[0083] If a short-time feature vector is needed, the short-time feature vector of the audio segment to be identified is extracted based on a preset speech feature extraction algorithm; that is, the feature vector of the short-time audio segment is the feature vector of the audio segment, for example, an audio segment extracted using an improved pyAudioAnalysis algorithm.

[0084] If a medium-time feature vector or a long-time feature vector is needed, the feature vectors of each short-time audio segment contained in the audio segment to be identified are obtained, and the medium-time or long-time feature vector of the audio segment to be identified is obtained based on a preset feature statistical algorithm. Medium-time audio segments and long-time audio segments include feature statistical values ​​of each short-time audio segment contained therein. For example, if a medium-time audio segment includes x short-time audio segments, then each feature value in the feature vector of the medium-time audio segment is a statistical index such as the mean, standard deviation, and maximum value of the corresponding feature values ​​in the x short-time audio segments. Similarly, for example, if a long-time audio segment includes y short-time audio segments, then each feature value in the feature vector of the long-time audio segment is a statistical index such as the mean, standard deviation, and maximum value of the corresponding feature values ​​in the y short-time audio segments.

[0085] The medium-time / long-time feature value can also be one or a combination of multiple features such as silence ratio, low-frequency energy ratio, high zero-crossing rate ratio, sub-band capacity ratio mean, spectrum centroid mean, band mean, spectrum transition, HMM probability output value, and other statistical feature values. These are usually used for classification and labeling of different types of audio, such as noise, speech, music, etc., and appropriate feature values ​​are set according to actual needs.

[0086] In this embodiment, the step S130 forms a large training set with different folders containing keywords, and inputs the improved pyAudioAnalysis for feature extraction. Here, according to the actual reasoning needs, each sample audio can be divided into one or more of short-time audio segments, medium-time audio segments and long-time audio segments. When divided into multiple different types of audio segments, the feature vectors of different audio segments do not need to be extracted at the same time. The number of eigenvalues ​​in the feature vector is designed according to the needs of the algorithm, and is not limited to the number used later. Here, the first 21 of pyAudioAnalysis and the design parameters are used as needed, and a total of 40 parameters are used. When calculating the medium-time and long-time eigenvalues, the mean and variance need to be calculated, so there will be 80 eigenvalues. In practical applications, audio segments of appropriate length can be selected according to the needs of the project for eigenvalue calculation. Finally, the eigenvalue matrix and the keyword label of each eigenvector are output. Each row / column eigenvalue is an eigenvector. Here, each row is an eigenvector. Each eigenvector contains 40 or 80 eigenvalues, that is, the number of columns of our matrix, and the number of rows is the total number of audio segments of all sample audios. The label is the keyword name or keyword number of the folder where each sample audio is located. Each feature vector corresponds to a keyword label. If a sample audio has multiple audio segments, then the feature vectors of all the audio segments of this sample audio have the same label, that is, the keyword label of the folder where the sample audio is located. Here, the number can be used instead of the keyword text itself as the keyword label. For example, if there are 9 keywords, the keyword labels are 1, 2, 3, 4, 5, 6, 7, 8, and 9 respectively.

[0087] In this embodiment, the training algorithm is an XGBoost model algorithm, and the keyword recognition model is an XGBoost model. In this embodiment, in step S130, the feature matrix data is made into a feature matrix of the input training algorithm according to the input data requirements of the image classification training algorithm, including the following steps:

[0088] Extract the feature vector of each sample audio segment of all sample audios, add the keyword label of the sample audio segment to the last column or the last row of the feature vector, and combine the feature vectors of each sample audio segment of all sample audios to obtain a feature matrix. In this embodiment, the feature vector is a row vector, and the keyword label is added to the last column.

[0089] Therefore, in step S130, the eigenvalue matrix of the sample audio and the keyword label of each eigenvector are combined to obtain the format of the input data of the XGBoost algorithm. Specifically, a column is added to the last column of the feature matrix, that is, the label corresponding to each eigenvector, so that each row matrix is ​​41-dimensional or 81-dimensional. Figure 3As shown, in step S130, in this embodiment, the feature vectors of each sample audio segment of all sample audios are combined to obtain a feature matrix, including the following steps:

[0090] S131: taking the feature vector of each sample audio segment as a row in a feature matrix;

[0091] S132: using a random number seed generator, disrupting the order of rows in the feature matrix where all eigenvectors are located, and obtaining an updated row order of all eigenvectors;

[0092] S133: Based on the updated row order of all eigenvectors, all eigenvectors are then combined to obtain a final feature matrix, which is input into the XGBoost training algorithm.

[0093] When training the keyword recognition model based on the feature matrix of the sample audio, similar to how XGBoost processes image classification data, a series of XGBoost training parameters are set, the training set and validation set are divided, the training and validation error rates are returned, and the XGBoost model is trained, and the training model and name are stored.

[0094] In this embodiment, after the step S110: establishing the training database, the following steps are also included:

[0095] Determine whether the duration of the sample audio is less than a preset duration threshold;

[0096] If so, the sample audio is repeated a specific number of times to obtain a new sample audio, and the new sample audio is used to replace the sample audio before being repeated the specific number of times, in order to obtain more effective feature values ​​of the sample.

[0097] For example, because the keyword is short, if the duration of a sample audio is very short, the corresponding sample audio can be repeated 2 to 5 times to obtain a new sample audio to replace the previous short sample audio, and then the sample audio is divided into sample audio segments and features are extracted. After obtaining the feature matrix, the keyword recognition model is trained, and the training effect is better. This can also reduce the problem of insufficient training data. After training the trained keyword recognition model, for very short audio to be recognized, there is no need to repeat it. It can be directly divided into audio segments and feature extracted to obtain the feature matrix, and then input into the keyword recognition model for recognition, which also has a high recognition accuracy rate.

[0098] By adopting the speech keyword recognition method, the classification effect of different keyword audios of the same speech is greatly improved, and the recognition accuracy rate reaches more than 98% to 99%. In the present invention, the data can be expanded to different dialect recognition. That is to say, a keyword is collected, and the pronunciation can be dialect, Mandarin, etc., as long as they are all placed in the same folder (the label of the same keyword), then after training, the model can recognize the pronunciation of the keyword in dialect, Mandarin, etc. The training data collected by the present invention can be a small amount, that is, 40 to 50 audio files for each keyword. Since the keyword is short, the corresponding wav can be repeated 2 to 5 times, and then used in training, the effect will be better, thus reducing the problem of less data. The speech keyword recognition method of the present invention is not only used for keyword classification in speech, but also for multi-class audio classification and labeling. At the same time, the speech keyword recognition method of the present invention is based on machine learning, the CPU has sufficient computing power, and there is no need to deploy a GPU. Of course, it can run on a GPU. The number of sample audio collections for each keyword is small, and effective training can also be carried out, which is different from the existing algorithms that require large-scale keyword collection.

[0099] like Figure 4 As shown, an embodiment of the present invention further provides a speech keyword recognition system, which is applied to the speech keyword recognition method, and the system includes:

[0100] The audio segmentation module M100 is used to establish a training database, wherein the training database includes sample audios of multiple keywords, divide each sample audio into multiple sample audio segments, and collect audio to be recognized and divide it into multiple audio segments to be recognized;

[0101] The feature extraction module M200 is used to extract the feature vectors of each sample audio segment, and make the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm; and extract the feature vectors of each audio segment to be identified, and make the feature vectors to be identified into a feature matrix suitable for input into the keyword recognition model;

[0102] A model training module M300 is used to perform model training based on the feature matrix of the input training algorithm and store the trained keyword recognition model;

[0103] The audio recognition module M400 is used to input the feature matrix to be recognized into the trained keyword recognition model to obtain the label of each audio segment to be recognized, and to count the labels of each audio segment to be recognized in the audio to be recognized, and to use the label with the largest number as the keyword label of the audio to be recognized.

[0104] The speech keyword recognition system of the present invention, during the training process, first collects sample audio through the audio segmentation module M100, divides the sample audio into multiple sample audio segments, then extracts the sample audio segments through the feature extraction module M200 and processes them into feature matrices that meet the requirements of the training algorithm, and performs model training through the model training module M300. In the feature extraction module M200 and the model training module M300, the algorithm commonly used for image classification is used on speech, and the audio data is retained, the data format is constructed, and then directly input into the algorithm. The keyword recognition model can be trained with a small amount of samples, the training time is short, and good recognition results can be achieved based on training with a small amount of data. The present invention has made an attempt to use the algorithm commonly used for image processing for speech processing, and it is not to generate a spectrogram from speech, and then use it as an image, and then use an image algorithm. Instead, it directly extracts data that effectively represents speech, such as speech feature values, from the speech audio file, and uses the algorithm for training, recognition and marking. The algorithm results show that the effect is very good.

[0105] During the recognition process, the audio to be recognized is first collected by using the audio segmentation module M100 and divided into multiple audio segments to be recognized, and then the feature extraction module M200 performs feature extraction on each audio segment to be recognized, and the feature vectors of each audio segment to be recognized are combined to obtain a feature matrix, and the keyword recognition model is used by the audio recognition module M400 to obtain the keyword label of each audio segment to be recognized, and by counting the keyword labels of all the audio segments to be recognized, the keyword label with the largest number is used as the keyword label of the entire audio to be recognized. The recognition method is simple, which greatly improves the accuracy of speech keyword recognition, has low algorithm complexity, and short recognition reasoning time.

[0106] In the speech keyword recognition system of the present invention, the function of each module can be implemented by adopting the specific implementation method of each step of the above-mentioned speech keyword recognition method. For example, the audio segmentation module M100 can adopt the specific implementation method of step S100 to perform audio segmentation, and divide the audio to be recognized into at least one of short-time audio segments, medium-time audio segments and long-time audio segments; the feature extraction module M200 can adopt the specific implementation method of step S200 to extract features of the audio segment; the model training module M300 can adopt the specific implementation method of step S300 to perform keyword model training and name and save it; the feature recognition module M400 can adopt the specific implementation method of step S400 to perform keyword tag recognition; the tag statistics module M500 can adopt the specific implementation method of step S500 to determine the keyword tag of the entire audio segment.

[0107] An embodiment of the present invention further provides a speech keyword recognition device, comprising a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to execute the steps of the speech keyword recognition method by executing the executable instructions.

[0108] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0109] Refer to the following Figure 5 The electronic device 600 according to this embodiment of the present invention is described. Figure 5 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0110] like Figure 5 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0111] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the electronic prescription circulation processing method section of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0112] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0113] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0114] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0115] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0116] By adopting the voice keyword recognition device provided by the present invention, the processor executes the voice keyword recognition method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned voice keyword recognition method.

[0117] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, which implements the steps of the voice keyword recognition method when executed by a processor. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above-mentioned electronic prescription circulation processing method section of this specification.

[0118] refer to Figure 6 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0119] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0120] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0121] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or cluster. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0122] By adopting the computer-readable storage medium provided by the present invention, the program stored therein implements the steps of the voice keyword recognition method when being executed, thereby obtaining the beneficial effects of the above-mentioned voice keyword recognition method.

[0123] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for recognizing speech keywords, characterized in that: It includes a training part and a recognition part, and the training part includes the following steps: Establishing a training database, wherein the training database includes sample audios of a plurality of keywords; Divide each sample audio into a plurality of sample audio segments, and use the keyword label of the sample audio as the keyword label of each sample audio segment; Extracting feature vectors of each sample audio segment, and making the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm, wherein each row or column of feature values ​​in the feature matrix corresponds to a feature vector; Performing model training based on the feature matrix of the input training algorithm, and storing the trained keyword recognition model; The identification part includes the following steps: Collecting audio to be recognized, dividing it into multiple audio segments to be recognized, and extracting feature vectors of each audio segment to be recognized; The feature vector to be identified is made into a feature matrix suitable for inputting into the keyword recognition model, and the trained keyword recognition model is inputted to obtain the label of each audio segment to be identified; Count the labels of each audio segment to be identified in the audio to be identified, and use the label with the largest number as the keyword label of the audio to be identified.

2. The method for speech keyword recognition according to claim 1, characterized in that: The training algorithm is XGBoost.

3. The method for speech keyword recognition according to claim 1, characterized in that: Dividing each sample audio into a plurality of sample audio segments, including dividing each sample audio into at least one of a short-time audio segment, a medium-time audio segment, and a long-time audio segment; Dividing the audio to be recognized into a plurality of audio segments to be recognized, including dividing each audio to be recognized into at least one of a short-term audio segment, a medium-term audio segment, and a long-term audio segment; The short-term audio segment, the medium-term audio segment, and the long-term audio segment are respectively an audio segment including a1 frames, an audio segment including a2 frames, and an audio segment including a3 frames. <a2<a3。 4. The method for speech keyword recognition according to claim 3, characterized in that: The step of extracting the feature vectors of each audio segment to be identified comprises the following steps: Determine the category of the required feature vector; If a short-time feature vector is needed, the short-time feature vector of the audio segment to be recognized is extracted based on a preset speech feature extraction algorithm; If a medium-time or long-time feature vector is needed, the feature vectors of each short-time audio segment contained in the audio segment to be identified are obtained, and the medium-time or long-time feature vector of the audio segment to be identified is obtained based on a preset feature statistical algorithm.

5. The method for speech keyword recognition according to claim 2, characterized in that: The feature matrix data is made into a feature matrix for input training algorithm according to the input data requirements of the image classification training algorithm, including the following steps: The feature vectors of each sample audio segment of all sample audios are extracted, and the keyword label of the sample audio segment is added to the last column or the last row of the feature vector, and the feature vectors of each sample audio segment of all sample audios are combined to obtain a feature matrix.

6. The method for speech keyword recognition according to claim 5, characterized in that: Combining the feature vectors of each sample audio segment of all sample audios to obtain a feature matrix includes the following steps: The feature vector of each sample audio segment is taken as a row in the feature matrix; Using a random number seed generator, the order of rows in the feature matrix where all eigenvectors are located is disrupted to obtain the updated row order of all eigenvectors; Based on the updated row order of all eigenvectors, all eigenvectors are then combined to get the final feature matrix, which is input into the XGBoost training algorithm.

7. The method for speech keyword recognition according to claim 5, characterized in that: After the training database is established, the following steps are also included: Determine whether the duration of the sample audio is less than a preset duration threshold; If so, the sample audio is repeated a specific number of times to obtain a new sample audio, and the new sample audio is used to replace the sample audio before being repeated the specific number of times.

8. A speech keyword recognition system, characterized in that: The method for speech keyword recognition applied to any one of claims 1 to 7, the system comprising: An audio segmentation module is used to establish a training database, wherein the training database includes sample audios of multiple keywords, divide each sample audio into multiple sample audio segments, and collect audio to be recognized and divide it into multiple audio segments to be recognized; A feature extraction module is used to extract feature vectors of each sample audio segment, and make the feature vectors into a feature matrix for input into the training algorithm according to the input data requirements of the image classification training algorithm; and extract feature vectors of each audio segment to be identified, and make the feature vectors to be identified into a feature matrix suitable for input into the keyword recognition model; A model training module, used for performing model training based on the feature matrix of the input training algorithm and storing the trained keyword recognition model; The audio recognition module is used to input the feature matrix to be recognized into the trained keyword recognition model to obtain the label of each audio segment to be recognized, and to count the labels of each audio segment to be recognized in the audio to be recognized, and to use the label with the largest number as the keyword label of the audio to be recognized.

9. A speech keyword recognition device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the speech keyword recognition method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the voice keyword recognition method according to any one of claims 1 to 7 are implemented.

Images