Acoustic model training method, speech recognition algorithm, storage medium and electronic device

Through the sliding window and slicing technology of the acoustic model training method, combined with the chain timing classification algorithm, the impact of slicing time slices on accuracy in speech recognition is solved, and stream recognition and accuracy improvement of speech recognition is achieved.

CN114822516BActive Publication Date: 2024-11-26SHENZHEN XISU TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210250296.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-11
Publication Date
2024-11-26
Estimated Expiration
2042-03-11

AI Technical Summary

Technical Problem

In the prior art, the sniping time slices during speech recognition have a great impact on the recognition accuracy, and the CTC method cannot strictly align the output phoneme time, resulting in phoneme deletion or repetition of the splicing results, which reduces the recognition accuracy.

Method used

A method of acoustic model training is proposed to obtain the audio feature map in the dataset, process it in segments using sliding window and slicing technology, and use chain timing classification algorithm to determine the effective speech recognition acoustic model, peeling and splicing segment prediction Results to obtain overall prediction results to ensure the accuracy of identification.

Benefits of technology

It realizes the audio splitting at will without changing the recognition result during speech recognition, improves the recognition accuracy, avoids phoneme loss or repetition at the splicing, and enhances the streaming recognition ability of speech recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114822516B_ABST
    Figure CN114822516B_ABST
Patent Text Reader

Abstract

The present invention discloses an acoustic model training method, a speech recognition algorithm, a storage medium and an electronic device. The acoustic model training method comprises the following steps: obtaining a first audio feature spectrum in a data set; obtaining an original acoustic model; sliding the first real-time audio feature spectrum according to a first preset time length to obtain a second real-time audio feature spectrum; dividing the first audio feature spectrum and the second audio feature spectrum according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows, respectively, and inputting the first segmented small windows and the second segmented small windows into the original acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results; stripping and splicing the above prediction results to obtain an overall prediction result; determining an effective speech recognition acoustic model according to the overall prediction result and a chain time series classification algorithm to solve the technical problem that the time slice division during speech recognition has a great influence on the recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to an acoustic model training method, a speech recognition algorithm, a storage medium and an electronic device. Background Art

[0002] Natural language embedded speech recognition technology is a high-tech technology that records audio in real time, extracts audio features, and transcribes it into corresponding text on a low-computing, low-resource embedded board on the end side. It is mainly used in real-time speech translation, voice interactive robots and other fields. The speech recognition process is roughly divided into audio preprocessing, acoustic model recognition, and language model recognition. Among them, acoustic model recognition is the key step to convert audio features into acoustic phonemes.

[0003] After the embedded device performs preliminary preprocessing on the real-time recorded audio, the obtained FBANK feature map is input into the acoustic model for calculation. The acoustic model will calculate and predict the acoustic phoneme prediction value corresponding to each frame feature based on the local features and long-term features of the FBANK feature map. In Chinese corpus recognition, the acoustic model is generally based on the phonemes of the initials and finals, or the pinyin without tones, or the pinyin with tones for phoneme modeling. Taking the modeling of the pinyin with tones as an example, the acoustic model will output the predicted pinyin with tones corresponding to the audio feature map.

[0004] Currently, acoustic models are generally based on deep learning methods, and the model structures used include CNN+CTC, DFSMN+CTC, CONFORMER, etc. Among them, due to the large amount of calculation of the CONFORMER model, it has not yet been applied to low-computing embedded devices. Currently, most of the commonly used end-side models are still based on the CTC model.

[0005] In conventional acoustic model training, the entire sentence corpus is used for training. In the corresponding inference process, it is also necessary to obtain a complete feature image in order to make predictions, which can only realize the function of speech-to-text conversion.

[0006] In real-time speech recognition scenarios, only acoustic information before the current moment can be obtained, so the ordinary acoustic model needs to be modified to adapt to the needs of streaming recognition, that is, the acoustic model calculates and predicts the phonemes of the corresponding frame through an acoustic feature map of limited length that only contains past information.

[0007] In the current prior art, in order to achieve streaming recognition, the speech is segmented into segments for segment recognition, and then the recognition results are spliced ​​together as the overall recognition result.

[0008] However, if the speech segmentation time slice is too short, it will cause insufficient context information and reduce the accuracy. If the speech segmentation time slice is too long, it will cause too long delay in speech recognition. In addition, since the CTC method cannot strictly align the output phoneme time, when splicing the results, it may cause missing or repeated phonemes at the splicing point, further reducing the recognition accuracy. Summary of the invention

[0009] The purpose of the present invention is to provide an acoustic model training method and device, a storage medium and an electronic device to solve the technical problem in the prior art that time slice segmentation during speech recognition has a significant impact on the recognition accuracy.

[0010] In order to achieve the above object, the present invention proposes an acoustic model training method, which comprises:

[0011] Obtaining a first audio feature spectrum in a data set;

[0012] Sliding the first real-time audio feature spectrum according to a first preset time length to obtain a second real-time audio feature spectrum;

[0013] The first audio feature spectrum and the second audio feature spectrum are respectively divided according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows;

[0014] Inputting the plurality of the first segmented small windows and the plurality of the second segmented small windows into the original acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results;

[0015] Separating and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result;

[0016] An effective speech recognition acoustic model is determined based on the overall prediction result and the chain temporal classification algorithm.

[0017] Optionally, the step of obtaining a first audio feature spectrum in the data set includes:

[0018] Get multiple raw audio data;

[0019] Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

[0020] Optionally, before the step of sliding the first real-time audio feature map according to a first preset time length to obtain a second real-time audio feature map, the step includes:

[0021] Performing random time shift and cepstrum mean variance normalization processing on the first audio feature spectrum data;

[0022] The first audio feature map is updated to the processed audio feature map.

[0023] Optionally, the step of stripping and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result includes:

[0024] Remove half of the first preset time length from the front end of each of the first segmented small window prediction results and the back end of each of the first segmented small window prediction results to obtain a plurality of first actual predicted speech sections;

[0025] Remove half of the first preset time length from the front end of each second segmented small window prediction result and the back end of each second segmented small window prediction result to obtain a plurality of second actual predicted speech sections;

[0026] Each of the first actual predicted speech sections and the second actual predicted speech sections are concatenated in chronological order to obtain an overall prediction result.

[0027] Optionally, the step of determining a valid speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises:

[0028] Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value;

[0029] When the probability value is within a first reliable preset range, confirming that the current training is effective;

[0030] The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

[0031] Optionally, after the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the step further includes:

[0032] When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

[0033] Optionally, before obtaining the original acoustic model, the method further includes:

[0034] Obtain the acoustic model for non-streaming recognition;

[0035] The non-streaming recognized acoustic model is used as the original acoustic model.

[0036] In order to achieve the above object, the present invention further proposes a speech recognition algorithm, the speech recognition algorithm comprising:

[0037] Acquire a first real-time audio feature map and an effective speech recognition acoustic model;

[0038] Sliding the first real-time audio feature spectrum according to a first preset time length to obtain a second real-time audio feature spectrum;

[0039] The first audio feature spectrum and the second audio feature spectrum are respectively divided according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows;

[0040] Inputting the plurality of the first segmented small windows and the plurality of the second segmented small windows into an effective speech recognition acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results;

[0041] Separating and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result;

[0042] Wherein, the effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0043] In order to achieve the above-mentioned purpose, the present invention also proposes a storage medium, in which at least one executable instruction is stored. When the executable instruction is executed on an electronic device, the electronic device executes the operation of the acoustic model training method as described above.

[0044] In order to achieve the above object, the present invention further proposes an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described above.

[0046] The present invention obtains the first audio feature spectrum in the data set; obtains the original acoustic model; obtains the second real-time audio feature spectrum by sliding the first real-time audio feature spectrum according to the first preset time length; respectively divides the first audio feature spectrum and the second audio feature spectrum according to the second preset time length to obtain multiple first segmented small windows and multiple second segmented small windows; inputs the multiple first segmented small windows and the multiple second segmented small windows into the original acoustic model for calculation to obtain multiple first segmented small window prediction results and multiple second segmented small window prediction results; strips and splices each of the first segmented small window prediction results and the second segmented small window prediction results to obtain the overall prediction result; determines the effective speech recognition acoustic model according to the overall prediction result and the chain time series classification algorithm. Through the above steps, the data used for training the model is processed and then used to train the effective speech recognition acoustic model, so that the effective speech recognition acoustic model finally obtained can realize the arbitrary time segmentation of the audio during speech recognition without changing its recognition result, thereby solving the technical problem that the time segmentation during speech recognition in the prior art has a greater impact on the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0048] Figure 1 The figure is a flowchart of an acoustic model training method in one embodiment.

[0049] Figure 2 Schematic diagram of a flow chart of a speech recognition algorithm in one embodiment.

[0050] Figure 3 A schematic diagram of a time period sliding window and segmentation of a speech recognition algorithm in one embodiment. DETAILED DESCRIPTION

[0051] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0052] In order to solve the technical problem in the prior art that time slice segmentation during speech recognition has a significant impact on recognition accuracy, the present invention proposes an acoustic model training method, a speech recognition algorithm, a storage medium and an electronic device.

[0053] In one embodiment, if Figure 1 As shown, the acoustic model training method includes:

[0054] S1. Obtain the first audio feature spectrum in the data set;

[0055] The data set is actually a training set established through existing speech data. At this time, the training set can contain one or more. The speech data can be in Chinese, Japanese, English and other languages. The training set can also be divided into various functional training sets such as common language and professional terms. This makes it convenient for users to train the acoustic model according to different needs to obtain a more accurate acoustic model for speech recognition after training. The audio feature spectrum is an image with time on the horizontal axis and frequency domain on the vertical axis.

[0056] S2, obtaining the original acoustic model;

[0057] At this time, the original acoustic model is selected from the acoustic models in the prior art.

[0058] S3, sliding the first real-time audio feature spectrum according to a first preset time length to obtain a second real-time audio feature spectrum;

[0059] Among them, the first preset time length can be set as needed, and the first audio feature spectrum is slid according to the first preset time length. The sliding window at this time means supplementing the entire time period of the second audio feature spectrum with the audio feature spectrum of the first preset time length, and deleting the audio feature spectrum of the first preset time length at the head, so as to obtain the second audio feature spectrum after the sliding window. At this time, the time length of the first audio feature spectrum is equal to the time length of the second audio feature spectrum.

[0060] S4. Segment the first audio feature graph and the second audio feature graph according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows;

[0061] Among them, the second preset time length can be set as needed. Since the audio feature spectrum is actually still a signal that changes according to time, after the first audio feature spectrum and the second audio feature spectrum are divided according to the second preset time length, multiple first segmented small windows and second segmented small windows with the same length can be obtained. At this time, the multiple segmentation results are multiple first segmented small windows and second segmented small windows with the same length. Through the data processing of the above steps, the integrity and compensability of the audio feature spectrum are guaranteed.

[0062] S5, inputting the plurality of the first segmented small windows and the plurality of the second segmented small windows into the original acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results;

[0063] At this time, multiple first segmented windows and multiple second segmented windows are input into the original acoustic model to perform feature extraction and model training. Each first segmented window and each second segmented window represent a complete speech. By inputting them into the original acoustic model, a segmented window prediction result can be obtained. When there are multiple segmented windows, multiple segmented window prediction results can be obtained.

[0064] S6, stripping and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result;

[0065] Since the segmented audio feature graphs in the previous step overlap in time length, the first segmented small window prediction result and the second segmented small window prediction result also have repeated segments. The stripping at this time is essentially to strip the repeated segments. Then the stripped segmented small window prediction results are spliced ​​in time order to obtain the overall prediction result. Figure 3 As shown, the following distances illustrate the peeling process:

[0066] The first preset time length is recorded as shift, and the second preset time length is recorded as T_frame. At this time, the time period of the first segmented small window prediction result should be (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_framen), and the time period of the second segmented small window prediction result should be (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_framen+shift), and so on. Taking (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_framen), (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_framen+shift) as an example, the stripping at this time is The overlapping parts among ((0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_framen), (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_framen+shift) in this process are deleted by one segment. For example, the segmented small window prediction result of the time period (X, T_frame1) is deleted from (0-T_frame1), and the segmented small window prediction result of the time period (shift-X, T_frame1+shift) is deleted from (shift, T_frame1+shift), or the time period of the overlapping parts of (0-T_frame1) and (shift, T_frame1+shift) are arbitrarily selected for deletion, so as to completely delete the overlapping time in the two time periods as the final result, that is, the technical scheme and technical effect of the present invention can be realized.

[0067] S7. Determine an effective speech recognition acoustic model based on the overall prediction result and the chain temporal classification algorithm.

[0068] When the acoustic model is actually trained, after selecting a suitable speech data training set according to its application scenario, the training set is processed as in S3 to S4 and then used for training, so that the suitable first preset time length and the second preset time length can be used according to the actual application scenario, and the multiple first segmented small windows and the multiple second segmented small windows after this processing are input into the original acoustic model for calculation to obtain multiple first segmented small window prediction results and multiple second segmented small window prediction results. At this time, since the first preset time length is repeated in the time of the segmented small windows, that is, the repetition of speech, the loss of a certain segment recognition error during the recognition process can be avoided, and since the segmentation Segments, different segmentations have different effects on recognition accuracy. At this time, there are combined recognitions of different segments in multiple first segment windows and multiple second segment windows, which can further improve the recognition accuracy. Finally, the prediction results of each first segment window and each second segment window are stripped and spliced ​​to obtain the overall prediction results to eliminate duplicate results, so as to splice into a complete overall prediction result. Finally, the effective speech recognition acoustic model is determined according to the overall prediction results and the chain time series classification algorithm. The parameters of the acoustic model can be adjusted so that the overall output of the effective speech recognition acoustic model is infinitely close to the speech signal represented by the audio feature map in the data set. Through the above scheme, when the effective speech recognition acoustic model after the above training is applied to speech recognition, the audio can be arbitrarily segmented in time without changing its recognition results, thereby solving the technical problem in the prior art that the segmentation of time slices during speech recognition has a greater impact on the recognition accuracy.

[0069] Optionally, the original acoustic model uses an original non-streaming recognition acoustic model, or a convolution-based acoustic model.

[0070] The original acoustic model is selected as the original non-streaming recognition acoustic model, or the convolution-based acoustic model. By combining it with the above acoustic model training method, the original non-streaming recognition acoustic model or the convolution-based acoustic model can be optimized and trained into a streaming recognition acoustic model, while keeping the amount of calculation and accuracy basically unchanged. Only the data processing method of the head and tail in the model during training needs to be modified, without increasing the labeling workload, and the optimization of the acoustic model can be achieved in a relatively simple way.

[0071] In one embodiment, the step of obtaining the first audio feature graph in the data set includes:

[0072] Get multiple raw audio data;

[0073] Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

[0074] The audio feature graph is an image with time on the horizontal axis and frequency domain on the vertical axis that you see with Audition (audio software) or Audacity (audio editing software). The original audio data is essentially an analog signal, and the audio feature graph can be obtained by processing the analog signal through the front end. Through the above steps, the audio feature graph in the data set can be standardized for easy training and recognition.

[0075] Optionally, taking the audio feature map as the Fbank audio feature map as an example, the process of preprocessing it is roughly divided into the following steps:

[0076] First, the collected speech is a time domain signal, which is pre-emphasized, framed, windowed, and Fourier transformed to obtain a frequency domain signal. Then, the frequency domain signal is subjected to amplitude square, Mel filtering, and logarithm to obtain the Fbank audio feature map. Pre-emphasis is to use a first-order high-pass filter to improve the signal-to-noise ratio of the signal in the high-frequency part.

[0077] In one embodiment, before the step of sliding the first real-time audio feature map according to a first preset time length to obtain a second real-time audio feature map, the step further includes:

[0078] Performing random time shift and cepstral mean variance normalization (CMVN) processing on the first audio feature spectrum data;

[0079] The first audio feature map is updated to the processed audio feature map.

[0080] It should be noted that before the first audio feature spectrum data is subjected to random time shift and cepstrum mean variance normalization (CMVN) processing, the originally collected audio data needs to be preprocessed. Taking the audio feature spectrum data as Fbank audio feature spectrum data as an example, the process of preprocessing it is roughly divided into the following steps:

[0081] First, the collected speech is a time domain signal, which is pre-emphasized, framed, windowed, and Fourier transformed to obtain a frequency domain signal. Then, the frequency domain signal is subjected to amplitude square, Mel filtering, and logarithm to obtain the Fbank audio feature spectrum data. Pre-emphasis is to use a first-order high-pass filter to improve the signal-to-noise ratio of the signal in the high-frequency part.

[0082] In the above embodiment, the audio feature spectrum data is subjected to random time shift and cepstral mean variance normalization (CMVN) processing, that is, the audio feature spectrum data after the above processing is processed, wherein random time shift refers to generating a random integer within a certain range and shifting the audio feature spectrum data by the value, and this method can effectively prevent overfitting of training. Cepstral mean variance normalization (CMVN) calculates the mean and variance of the feature spectrum and performs a normalization operation, which can enhance the robustness of the acoustic model.

[0083] In one embodiment, the step of stripping and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result includes:

[0084] Remove half of the first preset time length from the front end of each of the first segmented small window prediction results and the back end of each of the first segmented small window prediction results to obtain a plurality of first actual predicted speech sections;

[0085] Among them, there are examples in the previous article, such as Figure 3 As shown, the first preset time length is recorded as shift, and the second preset time length is recorded as T_frame_. At this time, the time period of the first segmented small window prediction result should be (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_frame3) and so on. Taking (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_frame3) as an example, the stripping at this time is to delete the part that overlaps with the second segmented small window prediction result in this process. For example, (0, T_frame1) is deleted. In addition to the first segmented small window prediction results of the time periods (0, x) and (x, T_frame1), delete the first segmented small window prediction results of the time periods (T_frame1, T_frame1+x) and (T_frame2-x, T_frame2) in (T_frame1, T_frame2), or delete the time periods (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_frame3) that overlap with the second segmented small window prediction results, and completely delete the overlapping time in the two time periods as the final result.

[0086] Remove half of the first preset time length from the front end of each second segmented small window prediction result and the back end of each second segmented small window prediction result to obtain a plurality of second actual predicted speech sections;

[0087] Among them, there is an example in the previous text, in which the first preset time length is recorded as shift, and the second preset time length is recorded as T_frame_. At this time, the time period of the second segmented small window prediction result should be (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_frame3+shift) and so on. Taking (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_frame3+shift) as an example, the stripping at this time is to separate (shift, T_frame1+shift), (T_frame1+shift, T_frame2+shift), (T_frame2+shift, T_frame3+shift) in this process from the time period of the first segmented small window prediction result. The segment should be the overlapping part between (0, T_frame1), (T_frame1, T_frame2), (T_frame2, T_frame3) minus one segment, for example, (shift, T_frame1+shift) deletes the first segmented small window prediction result of the time period (shift, shift+y) and (T_frame1+shift-y, T_frame1+shift), (T_frame1+shift, T_frame2+shift) deletes the second segmented small window prediction result of the time period (T_frame1+shift, T_frame1+shift+y) and (T_frame2+shift-y, T_frame2+shift), or both of them choose to delete the overlapping time period of (0-T_frame1) and (shift, T_frame1+shift), so as to completely delete the overlapping time of the two time periods as the final result.

[0088] Based on the above process, in this embodiment, x or y = half of the first preset time length, that is, x = shift / 2, y = shift / 2, (shift, T_frame1+shift) deletes the first segmented small window prediction results of the time period (shift, shift+shift / 2) and (T_frame1+shift-shift, T_frame1+shift), and (T_frame1+shift, T_frame2+shift) deletes the second segmented small window prediction results of the time period (T_frame1+shift, T_frame1+shift+shift / 2) and (T_frame2+shift-shift, T_frame2+shift). By limiting the stripping time length, when the deletion is limited to removing half of the first preset time length, it can be convenient to delete, and then a complex algorithm is used for calculation, which can reduce the security cost of this solution.

[0089] Each first actual predicted speech section and the second actual predicted speech section are concatenated in chronological order to obtain an overall prediction result.

[0090] Let's explain it in more detail with T_frame=1536ms and shift=T_frame / 2=768ms:

[0091] At this time, the time period of the first segmented small window prediction result is [0, 1536ms], [1536ms, 3072ms], [3072ms, 4608ms]

[0092] The time periods of the second segmented small window prediction results are [768ms, 2304ms], [2304ms, 3804ms], [3804ms, 5376ms].

[0093] That is, each segmented window T_frame = 1536ms, and each second segmented window slides shift = T_frame / 2 = 768ms relative to each first segmented window. Splitting and splicing means that the first segmented window takes out the result corresponding to the middle 768ms (384ms, 1152ms]), and the second segmented window also takes out the middle 768ms result, and then splices them together in sequence.

[0094] Through the above-mentioned splicing, fast stripping and splicing can be achieved, thereby achieving fast and complete splicing. In addition, possible phoneme missing or duplication at the splicing point can be avoided, further improving the recognition accuracy.

[0095] In one embodiment, the step of determining a valid speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises:

[0096] Determine the loss value and error rate of current training data according to the chain time series classification algorithm, the overall prediction result and the original acoustic model;

[0097] When the loss value is less than a first preset threshold value, and the error rate is lower than a second preset threshold value;

[0098] Confirm that the trained acoustic model is valid for speech recognition.

[0099] Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value;

[0100] Take the chain temporal classification algorithm as CTC loss as an example:

[0101] The input prediction value of CTC (Connectionist Temporal Classification) loss is the overall prediction result, and the input annotation value is the conventional annotation value of the entire corpus. Due to the characteristics of CTC, based on the original acoustic model, the result output by the overall model will automatically approach the correct annotation value. In this process, the loss value and error rate will be continuously obtained. Only when the loss value is less than the first preset threshold and the error rate is lower than the second preset threshold, can it be confirmed that the trained acoustic model is an effective speech recognition model, thereby achieving an unsupervised automatic alignment training effect. It should be noted that the first preset threshold and the second preset threshold at this time can be set by the user, or determined based on multiple experiments, and are not limited here.

[0102] When the probability value is within a first reliable preset range, confirming that the current training is effective;

[0103] The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

[0104] In one embodiment, after the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the step further includes:

[0105] When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

[0106] Through the above-mentioned chain time series classification algorithm, the results output by the overall model can be automatically approximated to the correct labeled values, thereby achieving an unsupervised automatic alignment training effect.

[0107] In one embodiment, before obtaining the original acoustic model, the method further includes:

[0108] Obtain the acoustic model for non-streaming recognition;

[0109] The non-streaming recognized acoustic model is used as the original acoustic model.

[0110] By taking the non-streaming recognition acoustic model as the original acoustic model and combining it with the above-mentioned acoustic model training method, it is possible to optimize and train the original non-streaming recognition acoustic model into a streaming recognition acoustic model while maintaining the main structure of the original model, without increasing the annotation workload, and only modifying the head and tail structures of the model during training, while keeping the computational effort and accuracy basically unchanged.

[0111] In order to solve the above problems, the present invention also proposes a speech recognition algorithm, such as Figure 2 As shown, the speech recognition algorithm includes:

[0112] S10, obtaining a first real-time audio feature spectrum and an effective speech recognition acoustic model;

[0113] The audio feature spectrum is an image with time as the horizontal axis and frequency domain as the vertical axis. The effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0114] S20, sliding the first real-time audio feature spectrum according to a first preset time length to obtain a second real-time audio feature spectrum;

[0115] S30, dividing the first audio feature spectrum and the second audio feature spectrum according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows;

[0116] It should be noted that the segmentation and sliding window processing at this time are based on the first preset time length and the second preset time length in the method of "segmenting the audio feature spectrum according to the first preset time length to obtain multiple segmented audio feature spectra, and sliding each segmented audio feature spectrum according to the second preset time length to obtain multiple segmented small windows" used when training the effective speech recognition acoustic model. After the training of the effective speech recognition acoustic model is completed, the first preset time length and the second preset time length used in the training can be deployed on the embedded end side. Thereby obtaining a matching accuracy effect.

[0117] S40, inputting the plurality of first segmented small windows and the plurality of second segmented small windows into an effective speech recognition acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results;

[0118] S50, stripping and splicing the first segmented small window prediction results and the second segmented small window prediction results to determine a recognition result corresponding to the real-time audio feature map;

[0119] It should be noted that the stripping at this time only needs to delete the repeated parts. Of course, the effect will be better if the 1 / 2 part of the first preset time length before and after each segmented small window recognition result is stripped.

[0120] In the above embodiment, the effective speech recognition acoustic model is determined according to the acoustic model training method as described above.

[0121] When the acoustic model is actually trained, after selecting a suitable speech data training set according to its application scenario, the training set is processed as in step S3 of the acoustic model training method and then used for training, so that a suitable first preset time length and a second preset time length can be used according to the actual application scenario, and the multiple segmented small windows after this processing are input into the original acoustic model for calculation to obtain multiple segmented small window prediction results. At this time, since the segmented small windows have a repetition of the first preset time length in time, that is, a repetition in speech, the loss of a certain segment due to recognition errors during the recognition process can be avoided, and the prediction results of the segmented small windows can be obtained by the repetition of the first preset time length in time. As for segmentation, different segmentations have different effects on recognition accuracy. At this time, there are combined recognitions of different segments in the previous and next segmented windows, which can further improve the recognition accuracy. Finally, the prediction results of each segmented window are stripped and spliced ​​to obtain the overall prediction results to eliminate duplicate results and splice into a complete overall prediction result. Finally, the effective speech recognition acoustic model is determined based on the overall prediction results and the chain time series classification algorithm. The parameters of the acoustic model can be adjusted so that the overall output of the effective speech recognition acoustic model is infinitely close to the speech signal represented by the audio feature map in the data set. Through the above scheme, when the effective speech recognition acoustic model after the above training is applied to speech recognition, the audio can be divided into time at will without changing its recognition results, thereby solving the technical problem in the prior art that the time slice division during speech recognition has a greater impact on the recognition accuracy. In addition, the real-time audio feature map is recognized based on the above principles, maintaining the model length and contextual relationship during training and inference, and cleverly utilizing the characteristics of CTC to achieve automatic annotation alignment, so that the splicing results can be spliced ​​in an orderly manner, achieving the effect of realizing streaming recognition while maintaining recognition accuracy as much as possible.

[0122] Optionally, on an embedded device, it is also necessary to use a second preset time length window length and a first preset time length sliding step length to feed the audio feature map into the effective speech recognition acoustic model frame by frame, and strip off 1 / 2 of the first preset time length before and after the output result, and output the recognition result of the current window in real time.

[0123] In order to solve the above problem, the present invention also proposes a storage medium, in which at least one executable instruction is stored. When the executable instruction is run on an electronic device, the electronic device executes the operation of the acoustic model training method as described above.

[0124] It should be noted that since the storage medium of the present application contains all the steps of the above-mentioned acoustic model training method, the storage medium can also implement all the schemes of the acoustic model training method and have the same beneficial effects, which will not be repeated here.

[0125] An acoustic model training method in the above method embodiment is performed. The device embodiment described above is merely illustrative, in which the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. It can be understood by those of ordinary skill in the art that all or some steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by the computer. In addition, it is well known to those skilled in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0126] In order to solve the above problem, the present invention further proposes an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0127] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described above.

[0128] It should be noted that since the electronic device of the present application includes all the steps of the above-mentioned acoustic model training method, the electronic device can also implement all the schemes of the acoustic model training method and have the same beneficial effects, which will not be repeated here.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An acoustic model training method, characterized in that: The acoustic model training method comprises: Obtaining a first audio feature spectrum in a data set; Get the original acoustic model; Sliding the first audio feature spectrum according to a first preset time length to obtain a second audio feature spectrum; The first audio feature spectrum and the second audio feature spectrum are respectively divided according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows; Inputting the plurality of the first segmented small windows and the plurality of the second segmented small windows into the original acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results; Separating and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result; An effective speech recognition acoustic model is determined based on the overall prediction result and the chain temporal classification algorithm.

2. The acoustic model training method according to claim 1, characterized in that: The step of obtaining the first audio feature spectrum in the data set includes: Get multiple raw audio data; Data processing is performed on the multiple original audio data to obtain a data set including multiple audio feature spectra.

3. The acoustic model training method according to claim 1, characterized in that: Before the step of sliding the first audio feature spectrum according to a first preset time length to obtain a second audio feature spectrum, the step further includes: Performing random time shift and cepstrum mean variance normalization processing on the first audio feature spectrum data; The first audio feature map is updated to the processed audio feature map.

4. The acoustic model training method according to claim 1, characterized in that: The step of stripping and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result comprises: Remove half of the first preset time length from the front end of each of the first segmented small window prediction results and the back end of each of the first segmented small window prediction results to obtain a plurality of first actual predicted speech sections; Remove half of the first preset time length from the front end of each second segmented small window prediction result and the back end of each second segmented small window prediction result to obtain a plurality of second actual predicted speech sections; Each of the first actual predicted speech sections and the second actual predicted speech sections are concatenated in chronological order to obtain an overall prediction result.

5. The acoustic model training method according to claim 1, characterized in that: The step of determining an effective speech recognition acoustic model according to the overall prediction result and the chained temporal classification algorithm comprises: Inputting the overall prediction result into a chain time series classification algorithm to obtain a probability value; When the probability value is within a first reliable preset range, confirming that the current training is effective; The original acoustic model is updated to the trained speech recognition acoustic model, and the trained speech recognition acoustic model is used as the effective speech recognition acoustic model.

6. The acoustic model training method according to claim 5, characterized in that: After the step of inputting the overall prediction result into the chain time series classification algorithm to obtain the probability value, the following step is further included: When the probability value is not within the first reliable preset range, the chain time series classification algorithm is continuously performed to obtain the probability value calculation, and the original acoustic model is retrained based on the data set.

7. The acoustic model training method according to claim 1, characterized in that: Before obtaining the original acoustic model, the method further includes: Obtain the acoustic model for non-streaming recognition; The non-streaming recognized acoustic model is used as the original acoustic model.

8. A speech recognition algorithm, characterized in that: The speech recognition algorithm includes: Acquire a first audio feature spectrum and an effective speech recognition acoustic model; Sliding the first audio feature spectrum according to a first preset time length to obtain a second audio feature spectrum; The first audio feature spectrum and the second audio feature spectrum are respectively divided according to a second preset time length to obtain a plurality of first segmented small windows and a plurality of second segmented small windows; Inputting the plurality of the first segmented small windows and the plurality of the second segmented small windows into an effective speech recognition acoustic model for calculation to obtain a plurality of first segmented small window prediction results and a plurality of second segmented small window prediction results; Separating and splicing each of the first segmented small window prediction results and the second segmented small window prediction results to obtain an overall prediction result; Wherein, the effective speech recognition acoustic model is determined according to the acoustic model training method as described in any one of claims 1-7.

9. A storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on the electronic device, the electronic device executes the operation of the acoustic model training method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the acoustic model training method as described in any one of claims 1-7.