Acoustic model training method, speech recognition algorithm, storage medium and electronic device

By segmenting and sliding windowing the audio feature map, and combining it with a chain-based temporal classification algorithm to optimize the acoustic model training, the impact of time slice segmentation on accuracy in speech recognition is resolved, thereby improving the accuracy and efficiency of streaming recognition.

CN114512122BActive Publication Date: 2024-11-26SHENZHEN XISU TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210103816.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-27
Publication Date
2024-11-26
Estimated Expiration
2042-01-27

AI Technical Summary

Technical Problem

In existing technologies, the segmentation of time slices during speech recognition has a significant impact on recognition accuracy. Furthermore, the CTC method cannot strictly align the output phoneme times, which may result in missing or repeated phonemes at the splicing points, thus reducing recognition accuracy.

Method used

By acquiring audio feature maps from the dataset, segmenting and processing them with sliding windows, and then inputting them into the original acoustic model, the prediction results of small sliding windows are extracted and spliced ​​together. Combined with a chain-like temporal classification algorithm, an effective speech recognition acoustic model is determined, and the training process is optimized to adapt to streaming recognition.

Benefits of technology

This method achieves the goal of improving speech recognition accuracy by eliminating the need to change the time segmentation of the recognition results during streaming recognition, thus avoiding the impact of speech segmentation time slices on recognition accuracy and maintaining the computational cost and accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114512122B_ABST
    Figure CN114512122B_ABST
Patent Text Reader

Abstract

The present invention discloses an acoustic model training method and device, a storage medium and an electronic device. The acoustic model training method comprises obtaining an audio feature spectrum in a data set; obtaining an original acoustic model; dividing the audio feature spectrum according to a first preset time length to obtain a plurality of segmented audio feature spectra, and sliding each of the segmented audio feature spectrum according to a second preset time length to obtain a plurality of small sliding windows; inputting the plurality of small sliding windows into the original acoustic model for calculation to obtain a plurality of small sliding window prediction results; stripping and splicing each of the small sliding window prediction results to obtain an overall prediction result; determining an effective speech recognition acoustic model according to the overall prediction result and a chain time series classification algorithm to solve the technical problem that the time slice division during speech recognition has a great influence on the recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to an acoustic model training method and device, a storage medium and an electronic device. Background Art

[0002] Natural language embedded speech recognition technology is a high-tech technology that records audio in real time, extracts audio features, and transcribes it into corresponding text on a low-computing, low-resource embedded board on the end side. It is mainly used in real-time speech translation, voice interactive robots and other fields. The speech recognition process is roughly divided into audio preprocessing, acoustic model recognition, and language model recognition. Among them, acoustic model recognition is the key step to convert audio features into acoustic phonemes.

[0003] After the embedded device performs preliminary preprocessing on the real-time recorded audio, the obtained FBANK feature map is input into the acoustic model for calculation. The acoustic model will calculate and predict the acoustic phoneme prediction value corresponding to each frame feature based on the local features and long-term features of the FBANK feature map. In Chinese corpus recognition, the acoustic model is generally based on the phonemes of the initials and finals, or the pinyin without tones, or the pinyin with tones for phoneme modeling. Taking the modeling of the pinyin with tones as an example, the acoustic model will output the predicted pinyin with tones corresponding to the audio feature map.

[0004] Currently, acoustic models are generally based on deep learning methods, and the model structures used include CNN+CTC, DFSMN+CTC, CONFORMER, etc. Among them, due to the large amount of calculation of the CONFORMER model, it has not yet been applied to low-computing embedded devices. Currently, most of the commonly used end-side models are still based on the CTC model.

[0005] In conventional acoustic model training, the entire sentence corpus is used for training. In the corresponding inference process, it is also necessary to obtain a complete feature image in order to make predictions, which can only realize the function of speech-to-text conversion.

[0006] In real-time speech recognition scenarios, only acoustic information before the current moment can be obtained, so the ordinary acoustic model needs to be modified to adapt to the needs of streaming recognition, that is, the acoustic model calculates and predicts the phonemes of the corresponding frame through an acoustic feature map of limited length that only contains past information.

[0007] In the current prior art, in order to achieve streaming recognition, the speech is segmented into segments for segment recognition, and then the recognition results are spliced ​​together as the overall recognition result.

[0008] However, if the speech segmentation time slice is too short, it will cause insufficient context information and reduce the accuracy. If the speech segmentation time slice is too long, it will cause too long delay in speech recognition. In addition, since the CTC method cannot strictly align the output phoneme time, when splicing the results, it may cause missing or repeated phonemes at the splicing point, further reducing the recognition accuracy. Summary of the invention

[0009] The purpose of the present invention is to provide an acoustic model training method and device, a storage medium and an electronic device to solve the technical problem in the prior art that time slice segmentation during speech recognition has a significant impact on the recognition accuracy.

[0010] In order to achieve the above object, the present invention proposes an acoustic model training method, which comprises:

[0011] Get the audio feature graph in the dataset;

[0012] Get the original acoustic model;

[0013] The audio feature spectrum is divided into multiple segmented audio feature spectra according to a first preset time length, and each segmented audio feature spectrum is windowed according to a second preset time length to obtain multiple small sliding windows;

[0014] Inputting the plurality of small sliding windows into the original acoustic model for calculation to obtain a plurality of small sliding window prediction results;

[0015] The prediction results of each small sliding window are stripped and spliced ​​to obtain the overall prediction result;

[0016] An effective speech recognition acoustic model is determined based on the overall prediction result and the chain temporal classification algorithm.

[0017] Optionally, the step of obtaining the audio feature graph in the data set includes:

[0018] Get multiple raw audio data;

[0019] Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

[0020] Optionally, before the step of obtaining a plurality of segmented audio feature spectra by dividing the audio feature spectra according to a first preset time length, and sliding each of the segmented audio feature spectra according to a second preset time length to obtain a plurality of small sliding windows, the step includes:

[0021] The audio feature spectrum data is subjected to random time shift and cepstrum mean variance normalization processing;

[0022] Update the audio feature map to the processed audio feature map.

[0023] Optionally, the step of stripping and splicing each of the small sliding window prediction results to obtain an overall prediction result includes:

[0024] Remove half of the second preset time length from the front end of each small sliding window prediction result and the back end of each small sliding window prediction result to obtain an actual predicted speech section;

[0025] Each of the actual predicted speech sections is concatenated to obtain an overall prediction result.

[0026] Optionally, the step of determining a valid speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises:

[0027] Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value;

[0028] When the probability value is within a first reliable preset range, confirming that the current training is effective;

[0029] The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

[0030] Optionally, after the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the step further includes:

[0031] When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

[0032] Optionally, before obtaining the original acoustic model, the method further includes:

[0033] Obtain the acoustic model for non-streaming recognition;

[0034] The non-streaming recognized acoustic model is used as the original acoustic model.

[0035] In order to achieve the above object, the present invention further proposes a speech recognition algorithm, the speech recognition algorithm comprising:

[0036] Obtain real-time audio feature maps and effective speech recognition acoustic models;

[0037] The real-time audio feature map is segmented and subjected to sliding window processing to obtain a plurality of small sliding windows;

[0038] Inputting a plurality of the small sliding windows into an effective speech recognition acoustic model to determine a plurality of small sliding window recognition results;

[0039] Determine the recognition result corresponding to the real-time audio feature graph after stripping and splicing the multiple small sliding window recognition results;

[0040] Wherein, the effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0041] In order to achieve the above-mentioned purpose, the present invention also proposes a storage medium, in which at least one executable instruction is stored. When the executable instruction is executed on an electronic device, the electronic device executes the operation of the acoustic model training method as described above.

[0042] In order to achieve the above object, the present invention further proposes an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0043] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described above.

[0044] The present invention obtains an audio feature spectrum in a data set; obtains an original acoustic model; obtains multiple segmented audio feature spectra by dividing the audio feature spectrum according to a first preset time length, and slides each segmented audio feature spectrum according to a second preset time length to obtain multiple small sliding windows; inputs the multiple small sliding windows into the original acoustic model for calculation to obtain multiple small sliding window prediction results; strips and splices each small sliding window prediction result to obtain an overall prediction result; and determines an effective speech recognition acoustic model based on the overall prediction result and a chain time series classification algorithm. Through the above steps, the data used to train the model is processed and then used to train an effective speech recognition acoustic model, so that the effective speech recognition acoustic model obtained at the end can achieve arbitrary time segmentation of the audio during speech recognition without changing its recognition result, thereby solving the technical problem in the prior art that the time segmentation during speech recognition has a greater impact on the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0046] Figure 1 The figure is a flowchart of an acoustic model training method in one embodiment.

[0047] Figure 2 FIG. 4 is a flow chart of a speech recognition algorithm in one of the embodiments. DETAILED DESCRIPTION

[0048] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0049] In order to solve the technical problem in the prior art that time slice segmentation during speech recognition has a significant impact on recognition accuracy, the present invention proposes an acoustic model training method and device, a storage medium and an electronic device.

[0050] In one embodiment, if Figure 1 As shown, the acoustic model training method includes:

[0051] S1. Obtain the audio feature graph in the dataset;

[0052] The data set is actually a training set established through existing speech data. At this time, the training set can contain one or more. The speech data can be in Chinese, Japanese, English and other languages. The training set can also be divided into various functional training sets such as common language and professional terms. This makes it convenient for users to train the acoustic model according to different needs to obtain a more accurate acoustic model for speech recognition after training. The audio feature spectrum is an image with time on the horizontal axis and frequency domain on the vertical axis.

[0053] S2, obtaining the original acoustic model;

[0054] At this time, the original acoustic model is selected from the acoustic models in the prior art.

[0055] S3, dividing the audio feature spectrum according to the first preset time length to obtain a plurality of segmented audio feature spectra, and sliding each of the segmented audio feature spectra according to the second preset time length to obtain a plurality of small sliding windows;

[0056] Among them, the first preset time length and the second preset time length can be set as needed. Since the audio feature spectrum is actually still a signal that changes in time, after the audio feature spectrum is divided according to the first preset time length, multiple segmented audio feature spectra with equal time lengths can be obtained. At this time, the multiple segmentation results are multiple segmented audio feature spectra with equal time lengths. Each segmented audio feature spectrum is windowed according to the second preset time length. At this time, the window means that the overall time period of the next segmented audio feature spectrum covers the end time of the previous segmented audio feature spectrum and extends forward by the second preset time length. The end time of the next segmented audio feature spectrum also extends forward by the second preset time length, thereby obtaining multiple small sliding windows, that is, the time of multiple small sliding windows covers the second preset time length of the time of the previous small sliding window. Through the data processing of the above steps, the integrity and compensability of the audio feature spectrum are guaranteed.

[0057] S4, inputting the multiple small sliding windows into the original acoustic model for calculation to obtain multiple small sliding window prediction results;

[0058] At this time, inputting multiple small sliding windows into the original acoustic model can perform feature extraction and model training. Each small sliding window represents a complete speech. Inputting it into the original acoustic model can obtain a small sliding window prediction result. When there are multiple small sliding windows, multiple small sliding window prediction results can be obtained.

[0059] S5, stripping and splicing each of the small sliding window prediction results to obtain an overall prediction result;

[0060] Since the segmented audio feature graphs in the previous steps overlap in time, the small sliding window prediction results also have repeated segments. The stripping at this time is essentially to strip the repeated segments. Then the stripped small sliding window prediction results are spliced ​​in chronological order to obtain the overall prediction results. The following distance illustrates the stripping process:

[0061] The first preset time length is recorded as T_frame, and the second preset time length is recorded as T_shift. At this time, the time period of the small sliding window prediction result should be (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift), (2T_frame-2T_shift, 3T_frame-2T_shift) and so on. Taking (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift) as an example, the stripping at this time is to strip (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift) in this process. e-T_shift), the overlapping part between (T_frame-T_shift, T_frame) is deleted for a section, for example, the small sliding window prediction result of the time period (T_frame-T_shift, X) is deleted from (0-T_frame), and the small sliding window prediction result of the time period (X, T_frame) is deleted from (2T_frame-T_shift), or the time period of the overlapping part of (0-T_frame) and (2T_frame-T_shift) is arbitrarily selected for deletion, so as to completely delete the overlapping time of the two time periods as the final result, that is, the technical solution and technical effect of the present invention can be realized.

[0062] S6. Determine an effective speech recognition acoustic model based on the overall prediction result and the chain temporal classification algorithm.

[0063] When the acoustic model is actually trained, after selecting a suitable speech data training set according to its application scenario, the training set is processed as in S3 and then used for training, so that the first preset time length and the second preset time length can be used according to the actual application scenario. The multiple small sliding windows after this processing are input into the original acoustic model for calculation to obtain multiple small sliding window prediction results. At this time, since there is a repetition of the second preset time length in the time of the small sliding window, that is, the repetition of speech, the loss of a certain segment due to recognition error during the recognition process can be avoided. Moreover, due to the segmentation, different segmentations have different effects on the recognition accuracy. At this time, there is a combined recognition of different segments in the previous and next small sliding windows, which can further improve the recognition accuracy. Finally, each of the small sliding window prediction results is stripped and spliced ​​to obtain the overall prediction result to eliminate repeated results, so as to splice into a complete overall prediction result. Finally, the effective speech recognition acoustic model is determined according to the overall prediction result and the chain time series classification algorithm. The parameters of the acoustic model can be adjusted so that the overall output of the effective speech recognition acoustic model is infinitely close to the speech signal represented by the audio feature map in the data set. Through the above scheme, when the effective speech recognition acoustic model trained as above is applied to speech recognition, the audio can be divided into time segments at will without changing its recognition result, thereby solving the technical problem in the prior art that the time segmentation during speech recognition has a significant impact on the recognition accuracy.

[0064] Optionally, the original acoustic model uses an original non-streaming recognition acoustic model, or a convolution-based acoustic model.

[0065] The original acoustic model is selected as the original non-streaming recognition acoustic model, or the convolution-based acoustic model. By combining it with the above acoustic model training method, the original non-streaming recognition acoustic model or the convolution-based acoustic model can be optimized and trained into a streaming recognition acoustic model, while keeping the amount of calculation and accuracy basically unchanged. Only the data processing method of the head and tail in the model during training needs to be modified, without increasing the labeling workload, and the optimization of the acoustic model can be achieved in a relatively simple way.

[0066] In one embodiment, the step of obtaining the audio feature graph in the data set includes:

[0067] Get multiple raw audio data;

[0068] Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

[0069] The audio feature graph is an image with time on the horizontal axis and frequency domain on the vertical axis that you see with Audition (audio software) or Audacity (audio editing software). The original audio data is essentially an analog signal, and the audio feature graph can be obtained by processing the analog signal through the front end. Through the above steps, the audio feature graph in the data set can be standardized for easy training and recognition.

[0070] Optionally, taking the audio feature map as the Fbank audio feature map as an example, the process of preprocessing it is roughly divided into the following steps:

[0071] First, the collected speech is a time domain signal, which is pre-emphasized, framed, windowed, and Fourier transformed to obtain a frequency domain signal. Then, the frequency domain signal is subjected to amplitude square, Mel filtering, and logarithm to obtain the Fbank audio feature map. Pre-emphasis is to use a first-order high-pass filter to improve the signal-to-noise ratio of the signal in the high-frequency part.

[0072] In one embodiment, the step of obtaining a plurality of segmented audio feature spectra by dividing the audio feature spectra according to a first preset time length, and sliding each of the segmented audio feature spectra according to a second preset time length to obtain a plurality of small sliding windows also includes:

[0073] The audio feature spectrum data is subjected to random time shift and cepstral mean variance normalization (CMVN) processing;

[0074] Update the audio feature map to the processed audio feature map.

[0075] It should be noted that before the audio feature spectrogram data is subjected to random time shift and cepstrum mean variance normalization (CMVN) processing, the originally collected audio data needs to be preprocessed. Taking the audio feature spectrogram data as Fbank audio feature spectrogram data as an example, the process of preprocessing the audio feature spectrogram data is roughly divided into the following steps:

[0076] First, the collected speech is a time domain signal, which is pre-emphasized, framed, windowed, and Fourier transformed to obtain a frequency domain signal. Then, the frequency domain signal is subjected to amplitude square, Mel filtering, and logarithm to obtain the Fbank audio feature spectrum data. Pre-emphasis is to use a first-order high-pass filter to improve the signal-to-noise ratio of the signal in the high-frequency part.

[0077] In the above embodiment, the audio feature spectrum data is subjected to random time shift and cepstral mean variance normalization (CMVN) processing, that is, the audio feature spectrum data after the above processing is processed, wherein random time shift refers to generating a random integer within a certain range and shifting the audio feature spectrum data by the value, and this method can effectively prevent overfitting of training. Cepstral mean variance normalization (CMVN) calculates the mean and variance of the feature spectrum and performs a normalization operation, which can enhance the robustness of the acoustic model.

[0078] In one embodiment, the step of stripping and splicing each of the small sliding window prediction results to obtain the overall prediction result includes:

[0079] Remove half of the second preset time length from the front end of each small sliding window prediction result and the back end of each small sliding window prediction result to obtain an actual predicted speech section;

[0080] Among them, there are examples in the previous text, in which the first preset time length is recorded as T_frame, and the second preset time length is recorded as T_shift. At this time, the time period of the small sliding window prediction result should be (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift), (2T_frame-2T_shift, 3T_frame-2T_shift) and so on. Taking (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift) as an example, the stripping at this time is to strip (0-T_frame), (T_frame-T_shift, 2T_frame-T_shift) in this process. The overlapping part between (0-T_shift, 2T_frame-T_shift) and (T_frame-T_shift, T_frame) is deleted for a section. For example, the small sliding window prediction result of the time period (T_frame-T_shift, X) is deleted from (0-T_frame), and the small sliding window prediction result of the time period (X, T_frame) is deleted from (2T_frame-T_shift), or the time period of the overlapping part of (0-T_frame) and (2T_frame-T_shift) is arbitrarily selected for deletion, so as to completely delete the overlapping time of the two time periods as the final result.

[0081] Based on the above process, in this embodiment, X=half of the second preset time length, that is, X=T_shift / 2, 0-T_frame) deletes the small sliding window prediction result of the time period (T_frame-T_shift, T_frame-T_shift / 2), and (T_frame-T_shift, 2T_frame-T_shift) deletes the small sliding window prediction result of the time period (T_frame-T_shift, T_frame / 2). By limiting the stripping time length, when the deletion is limited to removing half of the second preset time length, it can be convenient to delete, and then a complex algorithm is used for calculation, which can reduce the security cost of this solution.

[0082] Let's explain it in more detail with T_frame=1536ms and T_shift=T_frame / 2=768ms:

[0083] At this time

[0084] [0,1536ms],[768ms,2304ms],[1536ms,3072ms],..

[0085] That is, each window T_frame = 1536ms, and the next window slides relative to the current window by T_shift = T_frame / 2 = 768ms. The segmentation and splicing of S108 means that the result corresponding to the middle 768ms ([384ms, 1152ms]) is taken out from the first window, and the result of the middle 768ms is also taken out from the second window, and then they are spliced ​​together in sequence.

[0086] Each of the actual predicted speech sections is concatenated to obtain an overall prediction result.

[0087] Through the above-mentioned splicing, fast stripping and splicing can be achieved, thereby achieving fast and complete splicing. In addition, possible phoneme missing or duplication at the splicing point can be avoided, further improving the recognition accuracy.

[0088] In one embodiment, the step of determining a valid speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises:

[0089] Determine the loss value and error rate of current training data according to the chain time series classification algorithm, the overall prediction result and the original acoustic model;

[0090] When the loss value is less than a first preset threshold value, and the error rate is lower than a second preset threshold value;

[0091] Confirm that the trained acoustic model is effective for speech recognition.

[0092] Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value;

[0093] Take the chain temporal classification algorithm as CTC loss as an example:

[0094] The input prediction value of CTC (Connectionist Temporal Classification) loss is the overall prediction result, and the input annotation value is the conventional annotation value of the entire corpus. Due to the characteristics of CTC, based on the original acoustic model, the result output by the overall model will automatically approach the correct annotation value. In this process, the loss value and error rate will be continuously obtained. Only when the loss value is less than the first preset threshold and the error rate is lower than the second preset threshold, can it be confirmed that the trained acoustic model is an effective speech recognition model, thereby achieving an unsupervised automatic alignment training effect. It should be noted that the first preset threshold and the second preset threshold at this time can be set by the user, or determined based on multiple experiments, and are not limited here.

[0095] When the probability value is within a first reliable preset range, confirming that the current training is effective;

[0096] The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

[0097] In one embodiment, after the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the step further includes:

[0098] When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

[0099] Through the above-mentioned chain time series classification algorithm, the results output by the overall model can be automatically approximated to the correct labeled values, thereby achieving an unsupervised automatic alignment training effect.

[0100] In one embodiment, before obtaining the original acoustic model, the method further includes:

[0101] Obtain the acoustic model for non-streaming recognition;

[0102] The non-streaming recognized acoustic model is used as the original acoustic model.

[0103] By taking the non-streaming recognition acoustic model as the original acoustic model and combining it with the above-mentioned acoustic model training method, it is possible to optimize and train the original non-streaming recognition acoustic model into a streaming recognition acoustic model while maintaining the main structure of the original model, without increasing the annotation workload, and only modifying the head and tail structures of the model during training, while keeping the computational effort and accuracy basically unchanged.

[0104] In order to solve the above problems, the present invention also proposes a speech recognition algorithm, such as Figure 2 As shown, the speech recognition algorithm includes:

[0105] S10, obtaining real-time audio feature graph and effective speech recognition acoustic model;

[0106] The audio feature spectrum is an image with time as the horizontal axis and frequency domain as the vertical axis. The effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0107] S20, dividing the real-time audio feature map and performing sliding window processing to obtain a plurality of small sliding windows;

[0108] It should be noted that the segmentation and sliding window processing at this time are based on the first preset time length and the second preset time length in "segmenting the audio feature spectrum according to the first preset time length to obtain multiple segmented audio feature spectra, and sliding each segmented audio feature spectrum according to the second preset time length to obtain multiple small sliding windows" used when training the effective speech recognition acoustic model. After the training of the effective speech recognition acoustic model is completed, the first preset time length and the second preset time length used in the training can be deployed on the embedded end side. Thereby obtaining a matching accuracy effect.

[0109] S30, inputting the plurality of small sliding windows into an effective speech recognition acoustic model to determine a plurality of small sliding window recognition results;

[0110] S40, stripping and splicing the multiple small sliding window recognition results to determine the recognition result corresponding to the real-time audio feature map;

[0111] It should be noted that the stripping at this time only needs to delete the repeated part. Of course, the effect will be better if the 1 / 2 part of the second preset time length before and after each small sliding window recognition result is stripped.

[0112] In the above embodiment, the effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0113] When the acoustic model is actually trained, after selecting a suitable speech data training set according to its application scenario, the training set is processed as in step S3 of the acoustic model training method and then used for training, so that the appropriate first preset time length and second preset time length can be used according to the actual application scenario. The multiple small sliding windows after this processing are input into the original acoustic model for calculation to obtain multiple small sliding window prediction results. At this time, since there is a repetition of the second preset time length in the time of the small sliding window, that is, the repetition of speech, the loss of a certain segment due to recognition error during the recognition process can be avoided. Moreover, due to the segmentation, different segmentations have different effects on the recognition accuracy. At this time, there is a combined recognition of different segments in the previous and next small sliding windows, which can further improve the recognition accuracy. Finally, each of the small sliding window prediction results is stripped and spliced ​​to obtain the overall prediction result to eliminate the repeated results, so as to splice into a complete overall prediction result. Finally, the effective speech recognition acoustic model is determined according to the overall prediction result and the chain time series classification algorithm. The parameters of the acoustic model can be adjusted so that the overall output of the effective speech recognition acoustic model is infinitely close to the speech signal represented by the audio feature map in the data set. Through the above scheme, when the effective speech recognition acoustic model trained above is applied to speech recognition, the audio can be divided into time segments at will without changing its recognition result, thereby solving the technical problem in the prior art that the time segmentation during speech recognition has a significant impact on the recognition accuracy. In addition, the real-time audio feature graph is recognized based on the above principle, the model length and context relationship during training and inference are maintained, and the characteristics of CTC are cleverly used to achieve automatic annotation alignment, so that the splicing results can be spliced ​​in an orderly manner, achieving the effect of realizing streaming recognition while maintaining the recognition accuracy as much as possible.

[0114] Optionally, on an embedded device, it is also necessary to feed the audio feature map into the effective speech recognition acoustic model frame by frame with a first preset time length window length and a second preset time length sliding step length, and strip off 1 / 2 of the second preset time length before and after the output result, and output the recognition result of the current window in real time.

[0115] In order to solve the above problem, the present invention also proposes a storage medium, in which at least one executable instruction is stored. When the executable instruction is run on an electronic device, the electronic device executes the operation of the acoustic model training method as described above.

[0116] It should be noted that since the storage medium of the present application contains all the steps of the above-mentioned acoustic model training method, the storage medium can also implement all the schemes of the acoustic model training method and have the same beneficial effects, which will not be repeated here.

[0117] An acoustic model training method in the above method embodiment is performed. The device embodiment described above is merely illustrative, in which the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. It can be understood by those of ordinary skill in the art that all or some steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the computer. In addition, it is well known to those skilled in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0118] In order to solve the above problem, the present invention further proposes an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0119] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described above.

[0120] It should be noted that since the electronic device of the present application includes all the steps of the above-mentioned acoustic model training method, the electronic device can also implement all the schemes of the acoustic model training method and have the same beneficial effects, which will not be repeated here.

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An acoustic model training method, It is characterized in that The acoustic model training method comprises: Get the audio feature graph in the dataset; Get the original acoustic model; The audio feature spectrum is divided into multiple segmented audio feature spectra according to a first preset time length, and each segmented audio feature spectrum is windowed according to a second preset time length to obtain multiple small sliding windows; Inputting the plurality of small sliding windows into the original acoustic model for calculation to obtain a plurality of small sliding window prediction results; The prediction results of each small sliding window are stripped and spliced ​​to obtain the overall prediction result; An effective speech recognition acoustic model is determined based on the overall prediction result and the chain temporal classification algorithm.

2. The acoustic model training method according to claim 1, It is characterized in that The step of obtaining the audio feature graph in the data set includes: Get multiple raw audio data; Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

3. The acoustic model training method according to claim 1, It is characterized in that Before the step of dividing the audio feature spectrum into multiple segmented audio feature spectra according to the first preset time length, and sliding each segmented audio feature spectrum into multiple small sliding windows according to the second preset time length, the step further includes: The audio feature spectrum data is subjected to random time shift and cepstrum mean variance normalization processing; Update the audio feature map to the processed audio feature map.

4. The acoustic model training method according to claim 1, It is characterized in that The step of stripping and splicing each of the small sliding window prediction results to obtain the overall prediction result comprises: Remove half of the second preset time length from the front end of each small sliding window prediction result and the back end of each small sliding window prediction result to obtain an actual predicted speech section; Each of the actual predicted speech sections is concatenated to obtain an overall prediction result.

5. The acoustic model training method according to claim 1, It is characterized in that The step of determining an effective speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises: Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value; When the probability value is within a first reliable preset range, confirming that the current training is effective; The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

6. The acoustic model training method according to claim 5, It is characterized in that After the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the following step is further included: When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

7. The acoustic model training method according to claim 1, It is characterized in that Before obtaining the original acoustic model, the method further includes: Obtain the acoustic model for non-streaming recognition; The non-streaming recognized acoustic model is used as the original acoustic model.

8. A speech recognition algorithm, It is characterized in that The speech recognition algorithm includes: Obtain real-time audio feature maps and effective speech recognition acoustic models; The real-time audio feature map is segmented and subjected to sliding window processing to obtain a plurality of small sliding windows; Inputting a plurality of the small sliding windows into an effective speech recognition acoustic model to determine a plurality of small sliding window recognition results; Determine the recognition result corresponding to the real-time audio feature graph after stripping and splicing the multiple small sliding window recognition results; Wherein, the effective speech recognition acoustic model is determined according to the acoustic model training method as described in any one of claims 1-7.

9. A storage medium, It is characterized in that The storage medium stores at least one executable instruction, and when the executable instruction is executed on the electronic device, the electronic device executes the operation of the acoustic model training method as described in any one of claims 1 to 7.

10. An electronic device, It is characterized in that include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described in any one of claims 1-7.