A speech recognition method, apparatus, electronic device, and storage medium
By introducing a pre-defined acoustic model and a hard alignment algorithm into speech recognition, the problem of CTC's inability to accurately output timestamps was solved, and accurate time point output for each word in the speech recognition result was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING SINOVOICE TECH CO LTD
- Filing Date
- 2022-07-26
- Publication Date
- 2026-07-03
AI Technical Summary
In existing speech recognition technologies, the Connectionist Temporal Classification (CTC) method cannot accurately output the timestamp of the speech recognition text data, resulting in inaccurate time points.
By adding a preset acoustic model for alignment processing, and using hard alignment algorithms such as the Viterbi algorithm and the DNN-HMM model, combined with feature comparison processing, accurate timestamp information is generated.
It achieves accurate time point output for each word in the speech recognition result, improving the accuracy of timestamps.
Smart Images
Figure CN115394297B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of speech recognition, and more particularly to a speech recognition method, apparatus, electronic device, and storage medium. Background Technology
[0002] Automatic Speech Recognition (ASR) is a technology that studies how to recognize and convert human speech into text. It can be applied to services such as voice dialing, voice navigation, indoor device control, voice document retrieval, and simple dictation data entry. Existing technologies generally use Connectionist Temporal Classification (CTC) for speech recognition. However, CTC is a sequence recognition method. Although the representative frame of each recognized word can be used as the time point of the word, it cannot provide an accurate time point, which leads to the inability to output accurate timestamps when outputting speech recognition text data in actual situations. Summary of the Invention
[0003] To overcome the problems existing in related technologies, the present invention provides a speech recognition method, device, electronic device and storage medium.
[0004] According to a first aspect of the present invention, a speech recognition method is provided, the method comprising:
[0005] Acquire target audio data;
[0006] The target audio data is classified and identified to obtain the first text data;
[0007] The first text data is aligned using a preset acoustic model to obtain an alignment result, which includes timestamp information.
[0008] Target text data is generated based on the alignment result.
[0009] Optionally, the step of aligning the first text data using a preset processing model to obtain the alignment result includes:
[0010] Obtain the acoustic feature information corresponding to the first text data;
[0011] The first text data and the acoustic feature information are input into a preset acoustic model to obtain the alignment result.
[0012] Optionally, inputting the first text data and the acoustic feature information into the acoustic model includes:
[0013] The first text data and the acoustic feature information are input into a preset acoustic model for hard alignment.
[0014] Optionally, classifying and recognizing the target audio data to obtain the first text data includes:
[0015] The target audio data is input into a pre-set speech recognition model for classification and recognition, and the first text data is output.
[0016] Optionally, the step of inputting the first text data and the acoustic feature information into a preset acoustic model for hard alignment includes:
[0017] The first text data and the acoustic feature information are input into the acoustic model for feature comparison processing to generate timestamp information;
[0018] The timestamp information is matched one-to-one with each text data in the first text data.
[0019] According to a second aspect of the present invention, a voice recognition device is provided, the device comprising:
[0020] The acquisition module is used to acquire target audio data;
[0021] The recognition module is used to classify and recognize the target audio data to obtain the first text data;
[0022] An alignment module is used to align the first text data using a preset acoustic model to obtain an alignment result, the alignment result including timestamp information;
[0023] The output module is used to generate target text data based on the alignment result.
[0024] Optionally, the alignment module includes:
[0025] The acquisition unit is used to acquire acoustic feature information corresponding to the first text data;
[0026] The alignment unit is used to input the first text data and the acoustic feature information into the acoustic model to obtain the alignment result.
[0027] The alignment unit includes:
[0028] The alignment subunit is used to input the first text data and the acoustic feature information into a preset acoustic model for hard alignment processing.
[0029] According to a third aspect of the present invention, an electronic device is provided, comprising:
[0030] processor;
[0031] Memory used to store the processor's executable instructions;
[0032] The processor is configured to execute the instructions to implement the speech recognition method described in the first aspect of the embodiments of this application.
[0033] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the speech recognition method described in the first aspect of the present application.
[0034] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:
[0035] This invention can obtain target audio data; classify and identify the target audio data to obtain first text data; align the first text data using a preset acoustic model to obtain an alignment result; and generate target text data based on the alignment result. This invention adds an acoustic model for alignment, that is, aligning the identified result using a small, traditional acoustic model. Compared to existing speech recognition models, the text data obtained through alignment using the preset model can obtain accurate time points.
[0036] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description
[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0038] Figure 1 This is one of the flowcharts of the speech recognition method provided in the embodiments of this application;
[0039] Figure 2 yes Figure 1 The diagram shown is a schematic representation of the speech waveform in the speech recognition method provided in this embodiment of the application.
[0040] Figure 3 yes Figure 1 The diagram shown is a schematic representation of the predicted pronunciation waveform in the speech recognition method provided in this embodiment of the application.
[0041] Figure 4 This is the second flowchart of the speech recognition method provided in the embodiments of this application;
[0042] Figure 5 This is the third flowchart of the speech recognition method provided in the embodiments of this application;
[0043] Figure 6 It is a structural diagram of a voice recognition device provided by an embodiment of the present application;
[0044] Figure 7 It is a block diagram of an electronic device provided by an embodiment of the present application. Specific embodiments
[0045] Here, exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0046] It should be noted that CTC is used to solve the problem that it is difficult to correspond the input sequence and the output sequence one by one. The purpose is to directly learn the sequence data without事先标注好训练数据中输入序列和输入序列的映射关系 (previously annotating the mapping relationship between the input sequence and the input sequence in the training data), and achieve better results in sequence learning tasks such as voice recognition. In the general CTC method recognition, although the representative frame of each recognized character can be used as the time point of the character, since CTC is a sequence recognition method and cannot give an accurate time point, the actual situation is often not very accurate because the representative frame only represents that this frame best represents the phoneme, but not necessarily the start time of the phoneme. It should be noted that a phoneme is the smallest speech unit divided according to the natural attributes of speech. Analyzing according to the pronunciation actions in a syllable, one action constitutes one phoneme. Phonemes are divided into two categories: vowels and consonants. For example, in Chinese syllables, the syllable "ā" corresponding to the Chinese character "啊" has only one phoneme, the syllable "ài" corresponding to the Chinese character "爱" has two phonemes, and the syllable "dài" corresponding to the Chinese character "代" has three phonemes.
[0047] Therefore, in some scenario applications, in addition to outputting the result of voice recognition, it is also necessary to output the timestamp information of each character in the recognition result, that is, the start and end times of each character. Through a voice recognition method in an embodiment of the present application, accurate time point information can be output when performing voice recognition. Figure 1 It is a flowchart of a voice recognition method shown according to an exemplary embodiment, as Figure 1 shown, and includes the following steps:
[0048] Step 101, obtain target audio data.
[0049] It should be noted that in speech recognition, audio data is collected for the next step of speech recognition. The sound can be collected by any recording device or sound acquisition device. However, for a computer, to obtain the target audio data, the acquired sound data needs to be converted first.
[0050] Specifically, the collected sound data can be preprocessed at the front end. When the sound to be recognized is input, some optimization processing needs to be performed on the sound. For example, if there is a section of silence in the audio, the silent part needs to be cut off, so that the recognition can be more accurate. The Voice Activity Detection (VAD) technology can be used to detect the audio containing sound information and cut off the silent part. It should be noted that the silent detection duration can be set for the silent detection, and it is judged whether it is considered silent according to the duration and from what time to cut off. After the preprocessing is completed, acoustic feature parameters of the audio need to be extracted. These feature extractions mainly obtain the features of this audio in the form of parameters, and convert the features of the audio into speech feature vectors that can be processed by a computer, which is convenient for the computer to understand, record and compare. The feature parameters of each audio are basically different, and the audio features of the same passage with different timbres may be closer. The commonly used feature extraction parameters can be obtained through the following methods: Linear Predictive Cepstral Coefficient (LPCC), Mel Frequency Cepstrum Coefficient (MFCC), Linear Prediction Coefficients (LPC), Perceptual Linear Predictive (PLP). It should be noted that this application does not make specific limitations in this regard. Some audio contains noise and needs to be denoised to better perform the subsequent task process.
[0051] Step 102: Classify and recognize the target audio data to obtain the first text data.
[0052] After obtaining the target audio data in step 101, the target audio data is classified and recognized to obtain the first text data. Specifically, the target audio data is subjected to CTC speech recognition. For example, if the pronunciation of a voice is "Hello", the result of CTC speech recognition is also "Hello". It should be noted that for the training of the acoustic model of traditional speech recognition, for each frame of data, the corresponding label needs to be known to perform effective training. Before training the data, preprocessing of speech alignment needs to be done. And the process of speech alignment itself requires repeated iterations to ensure more accurate alignment, which is a time-consuming task in itself. Figure 2 yes Figure 1 The diagram shown below illustrates the pronunciation waveform in the speech recognition method provided in this application embodiment. Figure 2 The image shows a waveform diagram of the sound of the phrase "hello". Each box represents a frame of data. Traditional methods require knowing which phoneme each frame corresponds to. For example, frames 1, 2, 3, and 4 correspond to the sound of "n", frames 5, 6, and 7 to the sound of "i", frames 8 and 9 to the sound of "h", frames 10 and 11 to the sound of "a", and frame 12 to the sound of "o". Compared to traditional acoustic model training, acoustic model training using CTC as the loss function is a completely end-to-end acoustic model training. It does not require pre-alignment of the data; it only requires an input sequence and an output sequence for training. Therefore, there is no need for data alignment and labeling, and CTC directly outputs the probability of sequence prediction without external post-processing.
[0053] CTC speech is the result of an input sequence to an output sequence. For the CTC model, the main focus is on whether the predicted output sequence is close to (identical to) the real sequence, rather than whether each result in the predicted output sequence is exactly aligned with the input sequence at a given time point. Figure 3 yes Figure 1 The diagram shown below illustrates the predicted pronunciation waveform in the speech recognition method provided in this application embodiment. Figure 3 As shown, Figure 3 This is a diagram illustrating CTC prediction results. CTC introduces a "blank" (frame with no predicted value). Each predicted category corresponds to one spike in the entire speech segment; other non-spike positions are considered blank. For a speech segment, CTC's final output is a sequence of spikes, regardless of the duration of each phoneme. Figure 3 As shown, taking the pronunciation of "hello" as an example, the sequence result predicted by CTC may be slightly delayed in time from the actual pronunciation time point, and other time points will be marked as blank. Therefore, after CTC performs speech recognition on the target audio data, it will output the predicted sequence result, i.e., the first text data. In order to obtain a more accurate time point corresponding to the recognized speech text and reduce the delay, the operation in step 103 is performed.
[0054] Step 103: Align the first text data using a preset acoustic model to obtain an alignment result, which includes timestamp information.
[0055] In step 103, the recognition results from step 102 are aligned using a preset acoustic model to obtain the alignment results. Specifically, for example, the recognition result "Hello" and the corresponding acoustic features are fed into the acoustic model for hard alignment. By adding a traditional DNN-HMM alignment module combined with the hard alignment processing algorithm, accurate time point information can be obtained with a small increase in computation, that is, the alignment results include timestamp information.
[0056] Step 104: Generate target text data based on the alignment results.
[0057] In step 104, the final target text data after speech recognition is generated based on the final alignment. At this time, the target text data can contain the time point information of each word based on the timestamp information, that is, the start and end time of each word.
[0058] This invention can obtain target audio data; classify and identify the target audio data to obtain first text data; align the first text data using a preset acoustic model to obtain an alignment result; and generate target text data based on the alignment result. This invention adds an acoustic model for alignment, that is, aligning the identified result using a small, traditional acoustic model. Compared to existing speech recognition models, the text data obtained through alignment using the preset model can obtain accurate time points.
[0059] Figure 4 This is a second flowchart of the speech recognition method provided in the embodiments of this application, such as... Figure 4 As shown, it includes the following steps:
[0060] Step 101: Obtain the target audio data.
[0061] Step 102: Classify and identify the target audio data to obtain the first text data.
[0062] Steps 101-102 above are discussed in the preceding paragraphs and will not be repeated here.
[0063] Step 1031: Obtain the acoustic feature information corresponding to the first text data.
[0064] Step 1032: Input the first text data and acoustic feature information into the preset acoustic model to obtain the alignment result, which includes timestamp information.
[0065] Further, in step 1032, inputting the first text data and acoustic feature information into the acoustic model includes: inputting the first text data and the acoustic feature information into a preset acoustic model for hard alignment processing.
[0066] Specifically, the hard alignment process includes: inputting the first text data and the acoustic feature information into the acoustic model for feature comparison processing to generate timestamp information; and matching the timestamp information with each text data in the first text data one-to-one.
[0067] It should be noted that in this embodiment, the first text data is aligned using a preset acoustic model to obtain an alignment result. The alignment result includes timestamp information. Specifically, the process includes using the acoustic feature information corresponding to the first text data after CTC speech recognition. For example, the recognition result "Hello" and the corresponding acoustic features are fed into the acoustic model for Viterbi alignment. The Viterbi learning algorithm is a hard alignment, meaning that there is only a 0 or 1 belonging, i.e., a frame only belongs to a certain state. HMM decoding has two methods: the Viterbi algorithm and an approximation algorithm. In this embodiment, the Viterbi algorithm can be selected for hard alignment. The Viterbi algorithm is a dynamic programming algorithm. The Viterbi algorithm can obtain a backtracking path with the highest probability. Essentially, the Viterbi algorithm solves a multi-step optimal choice problem where each step involves multiple choices. For all possible choices at each step, the Viterbi algorithm saves the minimum total cost (or maximum value) from all previous steps to the current choice, as well as the choice of the previous step given the current cost. After calculating all the steps in sequence, the complete optimal choice path can be found by continuously searching for the previous step's choice through backtracking.
[0068] It should be noted that, in the embodiments of this application, the preset acoustic model can be aligned and identified by DNN+HMM. The DNN+HMM alignment method is similar to the GMM+HMM method, specifically, the DNN replaces the GMM in GMM+HMM for identification. The DNN records the emission probability information, while the transition matrix and initial state probability matrix still come from the HMM. The specific processing steps of the DNN-HMM acoustic model include: frame length segmentation and feature extraction, which can be performed using the Mel frequency cepstral coefficients (MFCC) method; Viterb alignment or Alignment alignment using the GMM-HMM acoustic model; clustering each frame (total number of phonemes) to obtain the probability of each frame belonging to each phoneme; decoding search using the HMM to obtain the optimal phoneme representation sequence for each frame; given the phoneme sequence, forced alignment is performed iteratively from GMM-HMM to DNN-HMM to DNN-HMM to obtain the alignment result.
[0069] Step 104: Generate target text data based on the alignment results.
[0070] Step 104 above is described in the preceding discussion and will not be repeated here.
[0071] This invention can obtain target audio data; classify and identify the target audio data to obtain first text data; align the first text data using a preset acoustic model to obtain an alignment result; and generate target text data based on the alignment result. This invention adds an acoustic model for alignment, that is, aligning the identified result using a small, traditional acoustic model. Compared to existing speech recognition models, the text data obtained through alignment using the preset model can obtain accurate time points.
[0072] Figure 5 This is the third flowchart of the speech recognition method provided in the embodiments of this application, as shown below. Figure 5 As shown, it includes the following steps:
[0073] Step 101: Obtain the target audio data.
[0074] Step 101 above is described in the preceding discussion and will not be repeated here.
[0075] Step 1021: Input the target audio data into a pre-set speech recognition model for classification and recognition, and output the first text data.
[0076] It should be noted that in this embodiment, CTC speech is the result of an input sequence to an output sequence. For the CTC model, the main focus is on whether the predicted output sequence is close to (identical to) the real sequence, rather than whether each result in the predicted output sequence is exactly aligned with the input sequence at a given time point. Figure 3 yes Figure 1 The diagram shown below illustrates the predicted pronunciation waveform in the speech recognition method provided in this application embodiment. Figure 3 As shown, Figure 3 This is a diagram illustrating CTC prediction results. CTC introduces a "blank" (frame with no predicted value). Each predicted category corresponds to one spike in the entire speech segment; other non-spike positions are considered blank. For a speech segment, CTC's final output is a sequence of spikes, regardless of the duration of each phoneme. Figure 3 As shown, taking the pronunciation of "hello" as an example, the sequence result predicted by CTC may be slightly delayed in time from the actual pronunciation time point, and other time points will be marked as blank. Therefore, after CTC performs speech recognition on the target audio data, it will output the predicted sequence result, i.e., the first text data. In order to obtain a more accurate time point corresponding to the recognized speech text and reduce the delay, the operation in step 103 is performed.
[0077] Step 103: Align the first text data using a preset acoustic model to obtain an alignment result, which includes timestamp information.
[0078] Step 104: Generate target text data based on the alignment results.
[0079] Steps 103-104 above are discussed in the preceding paragraphs and will not be repeated here.
[0080] This invention can obtain target audio data; classify and identify the target audio data to obtain first text data; align the first text data using a preset acoustic model to obtain an alignment result; and generate target text data based on the alignment result. This invention adds an acoustic model for alignment, that is, aligning the identified result using a small, traditional acoustic model. Compared to existing speech recognition models, the text data obtained through alignment using the preset model can obtain accurate time points.
[0081] Figure 6 This is a block diagram of a speech recognition device according to an exemplary embodiment. The device includes an acquisition module 601, a recognition module 602, an alignment module 603, and an output module 604.
[0082] Acquisition module 601 is used to acquire target audio data;
[0083] The recognition module 602 is used to classify and recognize the target audio data to obtain the first text data;
[0084] Alignment module 603 is used to align the first text data through a preset acoustic model to obtain an alignment result, the alignment result including timestamp information;
[0085] Output module 604 is used to generate target text data based on the alignment result.
[0086] Furthermore, the alignment module 603 includes:
[0087] The acquisition unit is used to acquire acoustic feature information corresponding to the first text data;
[0088] The alignment unit is used to input the first text data and the acoustic feature information into the acoustic model to obtain the alignment result.
[0089] Furthermore, the alignment unit includes:
[0090] The alignment subunit is used to input the first text data and the acoustic feature information into a preset acoustic model for hard alignment processing.
[0091] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0092] Figure 7 This is a block diagram illustrating an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0093] Reference Figure 7 The electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output interface 412, sensor component 414, and communication component 416.
[0094] Processing component 402 typically controls the overall operation of the device, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.
[0095] Memory 404 is configured to store various types of data to support the operation of electronic device 400. Examples of such data include instructions for any application or method operating on the device, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0096] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.
[0097] Multimedia component 408 includes a screen that provides an output interface between the electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0098] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.
[0099] Input / output interface 412 provides an interface between processing component 402 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.
[0100] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 may detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.
[0101] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other devices. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0102] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0103] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of an electronic device 400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0104] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.
[0105] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A voice recognition method, characterized by, The method includes: Acquire target audio data; The target audio data is classified and identified to obtain the first text data; The first text data and the corresponding acoustic feature information are aligned using a preset acoustic model to obtain an alignment result, which includes timestamp information. The timestamp information is generated by inputting the first text data and the corresponding acoustic feature information into the acoustic model for feature comparison processing. The timestamp information corresponds one-to-one with each text data in the first text data. The alignment process is a hard alignment, and each frame of the acoustic feature information belongs to a state. Target text data is generated based on the alignment result; wherein, the target text data includes the start and end times of each character; The acquisition of target audio data includes: collecting sound data, performing silence detection on the sound data, and removing silent portions from the sound data to complete the preprocessing of the sound data; the silent portions are determined and removed based on the silence detection duration; After the preprocessing of the sound data is completed, acoustic feature parameters are extracted from the sound data to obtain the target audio data; The alignment process includes: clustering each frame of the acoustic feature information to obtain the total number of phonemes; calculating the probability that each frame belongs to each phoneme based on the total number of phonemes; performing a decoding search based on the probability to obtain the optimal phoneme representation sequence for each frame; and iteratively performing forced alignment based on the optimal phoneme representation sequence and the likelihood value of the Gaussian mixture model to obtain the alignment result.
2. The method of claim 1, wherein, The step of aligning the first text data using a preset processing model to obtain the alignment result includes: Obtain the acoustic feature information corresponding to the first text data; The first text data and the acoustic feature information are input into a preset acoustic model to obtain the alignment result.
3. The method of claim 2, wherein, The step of inputting the first text data and the acoustic feature information into the acoustic model includes: The first text data and the acoustic feature information are input into a preset acoustic model for hard alignment.
4. The method of claim 1, wherein, The process of classifying and identifying the target audio data to obtain the first text data includes: The target audio data is input into a pre-set speech recognition model for classification and recognition, and the first text data is output.
5. A speech recognition apparatus characterized by comprising: The device includes: The acquisition module is used to acquire target audio data; The recognition module is used to classify and recognize the target audio data to obtain the first text data; An alignment module is used to align the first text data and the corresponding acoustic feature information using a preset acoustic model to obtain an alignment result. The alignment result includes timestamp information. The timestamp information is generated by inputting the first text data and the corresponding acoustic feature information into the acoustic model for feature comparison processing. The timestamp information corresponds one-to-one with each text data in the first text data. The alignment process is a hard alignment, and each frame of the acoustic feature information belongs to a state. An output module is used to generate target text data based on the alignment result; wherein the target text data includes the start and end times of each character; The acquisition of target audio data includes: collecting sound data, performing silence detection on the sound data, and removing silent portions from the sound data to complete the preprocessing of the sound data; the silent portions are determined and removed based on the silence detection duration; After the preprocessing of the sound data is completed, acoustic feature parameters are extracted from the sound data to obtain the target audio data; The alignment process includes: clustering each frame of the acoustic feature information to obtain the total number of phonemes; calculating the probability that each frame belongs to each phoneme based on the total number of phonemes; performing a decoding search based on the probability to obtain the optimal phoneme representation sequence for each frame; and iteratively performing forced alignment based on the optimal phoneme representation sequence and the likelihood value of the Gaussian mixture model to obtain the alignment result.
6. The apparatus of claim 5, wherein, The alignment module includes: The acquisition unit is used to acquire acoustic feature information corresponding to the first text data; The alignment unit is used to input the first text data and the acoustic feature information into the acoustic model to obtain the alignment result.
7. The apparatus of claim 6, wherein, The alignment unit includes: The alignment subunit is used to input the first text data and the acoustic feature information into a preset acoustic model for hard alignment processing.
8. An electronic device, comprising: include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement a speech recognition method as described in any one of claims 1 to 4.
9. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform a speech recognition method as described in any one of claims 1 to 4.