Long speech recognition method, device and medium

Abstract

The application relates to a long speech recognition method, device and medium, and belongs to the technical field of speech recognition. The method comprises the following steps: recognizing sub-speech data in an i-th period to obtain n i-th recognition results; combining an i-1-th candidate result and the i-th recognition result respectively to obtain n*m i-th combined results; determining the first m i-th candidate results in the i-th combined results; determining an i-th estimated optimal result in the i-th candidate results; in the case that the i-th estimated optimal result meets a preset output condition, acquiring x candidate results used when the i-th estimated optimal result is determined and outputting; determining estimated optimal results of subsequent periods by using x candidate results corresponding to the previous x periods respectively to obtain a recognition result of long speech data; the problem that the real-time output recognition result is not coherent due to the fact that the output optimal result is only a local optimal result of the current period, rather than the final recognition result of the entire long speech data, can be solved.

Description

[Technical Field]

[0001] This application relates to a long speech recognition method, device, and medium, belonging to the field of speech recognition technology. [Background Technology]

[0002] Long speech recognition technology refers to the technology of recognizing a user's speech to output a final recognition result. The long speech recognition process generally includes: inputting the speech data of the first segment of a long speech into a speech recognition model to obtain n first recognition results; inputting the n first recognition results into a search algorithm to determine the top m first recognition results with higher probabilities from the n first recognition results to obtain the first candidate results.

[0003] Next, the speech data from the second time segment of the long speech is input into the speech recognition model to obtain n second recognition results. These n second recognition results and m first candidate results are then input into a search algorithm, which combines them to obtain n×m combined results. The top m combinations with higher probabilities from these n×m combinations are selected as second candidate results. This process is repeated until the long speech recognition is complete. The candidate result with the highest probability from the m i-th candidate results is selected as the i-th optimal result. The first candidate results to the (i-1)-th candidate results used in calculating the i-th optimal result sequentially constitute the nodes on the optimal recognition path corresponding to the i-th optimal result. The nodes on the optimal recognition path corresponding to the i-th optimal result are the final output long speech recognition result. When real-time output of the recognition result is required, the optimal result for each time segment is output as the recognition result.

[0004] As can be seen from the above recognition process, starting from the second time period, the candidate results in each time period are obtained based on the candidate results of all previous time periods. Therefore, when the optimal result of each time period is output in real time, since the optimal result output is only the local optimal result of the current time period, rather than the final recognition result of the entire long speech data, the real-time output recognition results will be inconsistent, which will affect the post-processing problem after recognition and affect the user experience. [Summary of the Invention]

[0005] This application provides a long speech recognition method, device, and medium, which can solve the problem that the optimal output result is only the local optimal result for the current time period, rather than the final recognition result of the entire long speech data. This leads to inconsistent real-time output recognition results and affects post-processing. This application provides the following technical solution:

[0006] In a first aspect, a long speech recognition method is proposed, characterized in that the method includes:

[0007] Acquire long speech data collected by an audio acquisition device, wherein the long speech data includes sub-speech data in k time periods;

[0008] The sub-speech data within the first time segment of the long speech data is identified to obtain n first identification results; where n is a positive integer.

[0009] Determine the first m first candidate results from n first recognition results; where m is a positive integer less than or equal to n;

[0010] The sub-speech data within the i-th time period of the long speech data is identified to obtain n i-th identification results; where i is a positive integer taking values ​​from 2, and the value of i is less than k;

[0011] Combine the m (i-1)th candidate results and the n ith recognition results to obtain n×m ith combined results;

[0012] Determine the first m i-th candidate results from the n×m i-th combination results;

[0013] Determine the i-th estimated optimal result from the first m i-th candidate results;

[0014] If the estimated optimal result of the i-th time period satisfies the preset output conditions, obtain and output the x candidate results corresponding to the first x time periods used to determine the estimated optimal result of the i-th time period, where x is a positive integer less than or equal to i.

[0015] The optimal estimation result for each subsequent time period in the long speech data is determined by using the x candidate results corresponding to the first x time periods, and the recognition result of the long speech data is obtained.

[0016] Optionally, determining the optimal estimation result for each subsequent time period in the long speech data using the x candidate results corresponding to the first x time periods to obtain the recognition result of the long speech data includes:

[0017] For each value of y, the following steps are executed sequentially until i + y × j is greater than k, where y is a positive integer starting from 1:

[0018] The sub-speech data within the (i+y×j)th time period of the long speech data is identified to obtain n (i+y×j)th recognition results; j is a positive integer starting from 1, and i+y×j is less than k.

[0019] When j equals 1, the z-th (i+y×j-1)th candidate result and the n-th i-th recognition result are combined to obtain the n×z-th (i+y×j)th combined result; where z is a positive integer less than or equal to m, and the first y×x candidate results are used when determining the z-th (i+y×j-1)th candidate result;

[0020] Determine the first m (i+y×j)th candidate results from the n×z (i+y×j)th combination results;

[0021] When j is not equal to 1, the m i+y×j-1 candidate results and the n i-th recognition results are combined respectively to obtain n×m i+y×j combined results.

[0022] Determine the first m candidate results of the (i+y×j)th rank among the n×m combined results;

[0023] Determine the optimal estimated result for the (i+y×j)th rank among the first m candidate results;

[0024] If the optimal estimated result of the (i+y×j)th time period satisfies the preset output conditions, starting from the (y×x+1)th candidate result, obtain and output the x candidate results used when determining the optimal estimated result of the (i+y×j)th time period, and obtain the recognition results corresponding to the (y+1)×xth time period.

[0025] Optionally, after obtaining and outputting the x candidate results corresponding to the first x time periods used to determine the optimal result of the i-th estimation, the method further includes:

[0026] Identify the target time period for which the optimal estimation result has been determined but the identification result has not been output;

[0027] If the target time period meets the preset adjustment conditions, the preset output conditions are adjusted to output at least one recognition result within the target time period.

[0028] Optionally, the preset adjustment conditions include:

[0029] The number of characters indicated by the estimated optimal result corresponding to the target time period is greater than or equal to the first character number threshold;

[0030] or,

[0031] The duration of the target time period is greater than or equal to the duration threshold.

[0032] Optionally, the preset output conditions include:

[0033] The current time period number is a multiple of a preset value;

[0034] or,

[0035] The number of characters in the target time period for which the estimated optimal result has been determined but the recognition result has not been output is greater than or equal to the second character number threshold.

[0036] Optionally, when the preset output condition is that the number of characters corresponding to the target time period is greater than or equal to the second character number threshold, the step of obtaining x candidate results corresponding to the first x time periods used to determine the i-th estimated optimal result when the preset output condition is met includes:

[0037] Obtain the first i candidate results used when determining the optimal result of the i-th estimate;

[0038] The first L characters are determined from the first i candidate results;

[0039] Based on the Lth character, determine the x candidate results corresponding to the first x time periods respectively.

[0040] Optionally, based on the Lth character, x candidate results corresponding to the previous x time periods are determined, including:

[0041] Determine the x-th candidate result to which the L-th character belongs;

[0042] The xth candidate result and the previous x-1 candidate results are used to determine the x candidate results corresponding to the previous x time periods.

[0043] Optionally, the acquisition of long speech data collected by the audio acquisition device includes:

[0044] The starting endpoint of the long speech data is determined using a speech endpoint detection algorithm.

[0045] Based on the starting endpoint and the preset time interval, the sub-speech data for the k time periods are determined.

[0046] In a second aspect, an electronic device is provided, the self-moving device including a processor and a memory connected to the processor, the memory storing a program, and the processor executing the program to implement the long speech recognition method provided in the first aspect.

[0047] Thirdly, a computer-readable storage medium is provided, characterized in that the storage medium stores a program, which, when executed by a processor, is used to implement the long speech recognition method provided in the first aspect.

[0048] The beneficial effects of this application include at least the following: acquiring long speech data collected by an audio acquisition device; recognizing sub-speech data within a first time segment of the long speech data to obtain n first recognition results; determining the first m first candidate results from the n first recognition results; recognizing sub-speech data within the i-th time segment of the long speech data to obtain n i-th recognition results; combining the m (i-1)-th candidate results and the n i-th recognition results to obtain n×m i-th combined results; determining the first m i-th candidate results from the n×m i-th combined results; determining the i-th estimated optimal result from the first m i-th candidate results; and, if the i-th estimated optimal result satisfies a preset output condition, obtaining the confirmed result. The algorithm determines and outputs the x candidate results corresponding to the first x time periods when estimating the optimal result for the i-th time period. It then uses these x candidate results to determine the optimal estimated results for each subsequent time period in the long speech data, thus obtaining the recognition result for the long speech data. This approach addresses the problem that the output optimal result is only a local optimal result for the current time period, not the final recognition result for the entire long speech data, which leads to inconsistent real-time recognition results and affects post-processing. Since the x candidate results corresponding to the first x time periods tend to stabilize as i increases, outputting the x candidate results corresponding to the first x time periods each time improves the consistency of the recognition results output each time.

[0049] Meanwhile, since i is less than k, the recognition results for x time segments can be output before the long speech data ends, thus improving real-time performance.

[0050] In addition, if the target time period for which the optimal estimation result has been determined but no recognition result has been output meets the preset adjustment conditions, the preset output conditions can be adjusted to output at least one recognition result within the target time period. This can reduce the number of target time periods for which no recognition result has been output in the electronic device, reduce the occupation of storage resources, and improve flexibility.

[0051] In addition, since the number of characters contained in each time period is different, when using the preset value for the time period to output the recognition result, the time period with more characters will occupy too much storage resources and affect the running speed of the device, while the time period with fewer characters will affect the recognition accuracy. Therefore, using the number of characters as the preset output condition can ensure the accuracy and improve the running speed of electronic devices.

[0052] In addition, since the Lth character to be output may be located at the beginning or middle of the candidate results, outputting only the first L characters may result in the loss of the remaining character data of the candidate results corresponding to the Lth character. Therefore, using the Lth character to determine the x candidate results corresponding to the first x time periods respectively, and outputting x candidate results can avoid data loss and improve the recognition effect.

[0053] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. [Attached Image Description]

[0054] Figure 1 This is a flowchart of a long speech recognition method provided in one embodiment of this application;

[0055] Figure 2 This is a schematic diagram illustrating the output candidate results and pruning provided in one embodiment of this application;

[0056] Figure 3 This is a flowchart illustrating a specific example provided in one embodiment of this application;

[0057] Figure 4 This is a block diagram of an apparatus for a long speech recognition method provided in one embodiment of this application;

[0058] Figure 5 This is a block diagram of an electronic device provided in one embodiment of this application;

Detailed Implementation Methods

[0059] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0060] Optionally, this application uses the long speech recognition method provided in various embodiments in an electronic device as an example for illustration. The electronic device is a terminal or a server. The terminal can be a mobile phone, computer, tablet computer, scanner, electronic eye, or surveillance camera, etc. In actual implementation, the long speech recognition method can also be applied to other devices that are communicatively connected to the electronic device. This embodiment does not limit the type of electronic device.

[0061] The communication connection can be wired or wireless. The wireless communication method can be short-range communication or wireless communication, etc. This embodiment does not limit the communication method between the self-moving device and other devices.

[0062] Figure 1 This is a flowchart of a long speech recognition method provided in one embodiment of this application. The method includes at least the following steps:

[0063] Step 101: Acquire long speech data collected by the audio acquisition device.

[0064] The long speech data includes k time-segment sub-speech data.

[0065] Audio acquisition devices are used to collect voice data. Audio acquisition devices include, but are not limited to, video conferencing cameras, image capture cards, or personal computer (PC) microphones. In practice, audio acquisition devices can be integrated into electronic devices or can be other devices that communicate with electronic devices. This embodiment does not limit the implementation method of the audio acquisition device.

[0066] Long-duration audio data refers to audio data with a relatively long duration or that is collected in real time. Examples include long-duration conference audio data or user voice data collected in real time by a smart terminal.

[0067] When collecting long speech data, since the computational cost of detecting the starting point of the speech data is much smaller than that of speech recognition, in order to reduce the computational cost of subsequent speech recognition, it is usually necessary to obtain the starting point of the long speech data, that is, to obtain the starting point of the sub-speech data of the first time segment in the long speech data.

[0068] To illustrate, acquiring long speech data collected by an audio acquisition device includes: using a speech endpoint detection algorithm to determine the starting endpoint of the long speech data; and determining k sub-speech data segments based on the starting endpoint and a preset time interval.

[0069] Voice Activity Detection (VAD), also known as speech boundary detection, is used to accurately locate the start and end points of speech data in noisy speech, and remove useless parts such as silence and noise from the speech data to obtain the effective part of the speech data.

[0070] It should be noted that when the speech endpoint detection algorithm determines that the speech data segment is a useless part such as silence or noise, it will remove that part; when the speech endpoint detection algorithm determines that the speech data segment is a human voice, it will retain that part.

[0071] Speech detection algorithms include, but are not limited to: threshold-based VAD, classifier-based VAD, or model-based VAD.

[0072] Threshold-based VAD (Voice over Action) extracts time-domain or frequency-domain features and sets appropriate thresholds to distinguish between speech and non-speech data, accurately locating the start and end points of speech data.

[0073] Classification-based VAD treats speech detection as a binary classification problem between speech and non-speech, and then trains a classifier using machine learning methods to distinguish between speech and non-speech, and accurately locate the start and end points of speech data.

[0074] Model-based VAD involves constructing a complete acoustic model to distinguish between speech and non-speech data, accurately locating the start and end points of speech data. For example, a Hidden Markov Model (HMM) can be used to detect long speech data and locate its start and end points.

[0075] Step 102: Recognize the sub-speech data within the first time period of the long speech data to obtain n first recognition results.

[0076] Where n is a positive integer.

[0077] Speech recognition refers to the process of converting long speech data into text information.

[0078] Speech recognition models include: end-to-end recognition models with an encoder-decoder architecture or traditional speech recognition models based on acoustic models, pronunciation dictionaries, speech models, and decoders.

[0079] The encoder-decoder architecture is a general framework where the encoder and decoder parts can be any data such as speech or text. The model can be a recurrent neural network (RNN), a gate recurrent unit (GRU), a convolutional neural network (CNN), or a long short-term memory (LSTM), etc. This embodiment does not limit the specific implementation of the end-to-end recognition model.

[0080] Optionally, the first recognition result includes the recognized words and the probability corresponding to each word. A higher probability word indicates higher accuracy in the current time period. In this embodiment, the probability is the posterior probability.

[0081] Step 103: Determine the top m first candidate results from the n first recognition results.

[0082] Where m is a positive integer less than or equal to n.

[0083] The probability is used to determine the top m first candidate results from n first recognition results, including: based on the probability of the first recognition results, determining the top m first candidate results with the highest probability from n first recognition results.

[0084] Since the candidate results for subsequent time periods are generated based on the candidate results for the current time period, retaining m first candidate results can increase the probability of candidate results for subsequent time periods and expand the search range of candidate results for subsequent time periods.

[0085] Step 104: Recognize the sub-speech data in the i-th time period of the long speech data to obtain n i-th recognition results.

[0086] Where i is a positive integer starting from 2, and the value of i is less than k.

[0087] For example, taking an end-to-end recognition model with an encoder-decoder architecture as an example, the sub-speech data is input into the model to extract frequency domain features. The encoder converts the frequency domain feature sequence into a fixed-dimensional vector. The decoder generates a word probability distribution vector based on the encoder's output and the decoder's output in the previous time period to represent the recognized word and the probability distribution of the recognized word. That is, the word probability distribution vector is the recognition result at the current time.

[0088] The vector dimension of the vocabulary probability distribution vector is the length of the vocabulary.

[0089] It is important to note that since i cannot be 1, when the sub-speech data input to the end-to-end recognition model of the encoder-decoder architecture is the sub-speech data of the first time segment, the input to the decoder is the output of the encoder and the identifier. <bos>Here, "bos" stands for "begin of sequence," and in the model, it represents the start.

[0090] Step 105: Combine the m (i-1)th candidate results and the n ith recognition results to obtain n×m ith combined results.

[0091] Step 106: Determine the first m i-th candidate results from the n×m i-th combination results.

[0092] Step 107: Determine the i-th estimated optimal result from the first m i-th candidate results.

[0093] When the current time period is the i-th time period, the i-th optimal identification sequence, which is composed of the first i candidate results of generating the i-th estimated optimal result, is the optimal identification sequence for the current i-th time period.

[0094] Step 108: If the estimated optimal result of the i-th time period satisfies the preset output conditions, obtain and output the x candidate results corresponding to the first x time periods used to determine the estimated optimal result of the i-th time period.

[0095] Where x is a positive integer less than or equal to i.

[0096] To illustrate, the preset output conditions include the current time period number being a multiple of a preset value.

[0097] The preset value can be input by the user or obtained from other devices. This embodiment does not limit the implementation method of the preset value.

[0098] In long speech data, time segments that are closer in time are more closely connected, while those that are farther apart in time are less connected. Therefore, the x candidate results corresponding to the first x time segments in the optimal recognition sequence for each time segment tend to stabilize. That is, the i-th optimal recognition sequence and the (i-1)-th recognition sequence have the same x candidate results corresponding to the first x time segments. At this point, the difference between the output x candidate results corresponding to the first x time segments and the output x candidate results corresponding to the first x time segments after all the long speech data has been recognized is small, thus ensuring the accuracy of the output x candidate results corresponding to the first x time segments.

[0099] Optionally, if the preset output condition is that the time period number of the current time period is a multiple of a preset value, and if the optimal estimated result of the i-th time period satisfies the preset output condition, after obtaining and outputting the x candidate results corresponding to the first x time periods used to determine the optimal estimated result of the i-th time period, the method further includes: pruning other candidate results other than the candidate results already output in the first x time periods.

[0100] For example, such as Figure 2 As shown, when x=3, the fifth optimal estimate for time period t5 is F2. The first three candidate results indicated by F2 are B1, C1 and D2, respectively. After outputting B1, C1 and D2, the other results in the first three time periods other than the candidate results that have been output are pruned. In other words, at this time, E2 and E3 generated based on B1, C1 and D2 are retained in time period t4, while F2 and F3 generated based on B1, C1 and D2 are retained in time period t5.

[0101] By pruning other candidate results from the first x time periods, the occupied space can be freed up, thereby reducing the storage resources required for long speech recognition and improving the operating speed of electronic devices.

[0102] Because different users speak at different speeds and the requirements of different occasions vary, the number of characters contained in the candidate results at different times in different long speech data is different.

[0103] Therefore, the preset output conditions may also include the number of characters corresponding to the target time period for which the estimated optimal result has been determined but the recognition result has not been output, which is greater than or equal to the second character number threshold.

[0104] The second character count threshold can be input by the user or obtained from other devices. This embodiment does not limit the implementation method of the second character count threshold.

[0105] When the number of characters reaches the second character threshold, outputting x candidate results corresponding to the first x time periods can avoid the problem of excessive accumulation by electronic devices.

[0106] When determining the x candidate results corresponding to the first x time periods by the number of characters, x will change with the number of characters corresponding to each candidate result. Therefore, under the preset output condition that the number of characters corresponding to the target time period is greater than or equal to the second character number threshold, and under the condition that the estimated optimal result of the i-th time period meets the preset output condition, the x candidate results corresponding to the first x time periods used to determine the estimated optimal result of the i-th time period are obtained, including: obtaining the first i candidate results used to determine the estimated optimal result of the i-th time period; determining the first L characters in the first i candidate results; and determining the x candidate results corresponding to the first x time periods based on the L-th character.

[0107] Since the Lth character may not be the last character in the candidate results to which the Lth character belongs, outputting the first L characters may result in the loss of characters after the Lth character in the candidate results to which the Lth character belongs.

[0108] Optionally, determining x candidate results corresponding to the first x time periods based on the Lth character includes: determining the xth candidate result to which the Lth character belongs; and determining x candidate results corresponding to the first x time periods based on the xth candidate result and the first x-1 candidate results.

[0109] Alternatively, based on the Lth character, determine the x candidate results corresponding to the previous x time periods, including: determining the candidate result to which the Lth character belongs; determining the number of characters in the candidate result to which the Lth character belongs and the position number of the Lth character in the candidate result; if the ratio of the position number to the number of characters in the candidate result to which the Lth character belongs is greater than or equal to a preset percentage, determine the xth candidate result to which the Lth character belongs; and determine the x candidate results corresponding to the previous x time periods based on the xth candidate result and the previous x-1 candidate results.

[0110] If the ratio of the position number to the number of characters in the candidate results to which the Lth character belongs is less than a preset percentage, determine the (x+1)th candidate result to which the Lth character belongs; determine the x candidate results corresponding to each of the previous x time periods based on the xth candidate result and the x candidate results corresponding to each of the previous x time periods.

[0111] The preset percentage can be 50% or 60%, and this embodiment does not limit the implementation method of the preset percentage.

[0112] Step 109: Use the x candidate results corresponding to the first x time periods to determine the optimal estimation results for each subsequent time period in the long speech data, and obtain the recognition results of the long speech data.

[0113] Since the accuracy of the x candidate results corresponding to the first x time periods is relatively high, the accuracy of the estimated optimal results for each subsequent time period in the long speech data determined based on the x candidate results corresponding to the first x time periods can be guaranteed.

[0114] Specifically, steps 105-109 can be implemented using the beam search algorithm. In actual implementation, steps 105-109 can also be implemented using the Viterbi algorithm. This embodiment does not limit the specific implementation method of steps 105-109.

[0115] Optionally, the optimal estimation results for each subsequent time period in the long speech data are determined using the x candidate results corresponding to the first x time periods, to obtain the recognition results of the long speech data. This includes: for each value of y, the following steps are executed sequentially until i+y×j is greater than k: the sub-speech data in the i+y×j time period of the long speech data are recognized to obtain n i+y×j recognition results; when j equals 1, the z i+y×j-1 candidate results and the n i-th recognition results are combined to obtain n×z i+y×j combined results; z is a positive integer less than or equal to m, and the previously output y×x candidate results are used when determining the z i+y×j-1 candidate results; in the n×z time periods... From the (i+y×j)th combination results, determine the first m (i+y×j)th candidate results; if j is not equal to 1, combine the m (i+y×j-1)th candidate results and the n (i)th recognition results respectively to obtain n×m (i+y×j)th combination results; determine the first m (i+y×j)th candidate results from the n×m (i+y×j)th combination results; determine the (i+y×j)th estimated optimal result from the first m (i+y×j)th candidate results; if the (i+y×j)th estimated optimal result satisfies the preset output conditions, starting from the (y×x+1)th candidate result, obtain and output the x candidate results used when determining the (i+y×j)th estimated optimal result, and obtain the recognition results corresponding to the (y-1)×x+1th time period to the (y×x)th time period.

[0116] Where y is a positive integer starting from 1, j is a positive integer starting from 1, and i + y × j is less than k. Illustratively, the maximum value of i is the same as the maximum value of j.

[0117] For example, with n = 100, m = 4, z = m (i.e., z = 4), i = 5, j = 5, y = 1, k = 18, x = 3, and the preset output condition is that the current time period number is a multiple of the preset value, with the preset value being 5.

[0118] Since 5 + 1 × 5 is less than 18, the sub-speech data of the 5 + 1 × 5 time period, i.e. the 10th time period, is recognized to obtain 100 10th recognition results. The 4 9th candidate results and the 100 10th recognition results are combined to obtain 400 10th combination results. The first 4 10th candidate results are determined from the 400 10th combination results. The 10th estimated optimal result is determined from the first 4 10th candidate results. The time period number 10 corresponding to the 10th estimated optimal result is a multiple of 5. Therefore, starting from the 4th candidate result, the 3 candidate results used to determine the 10th estimated optimal result are obtained and output to obtain the recognition results corresponding to the 4th to 6th time periods.

[0119] By looping once for every maximum value of j, and determining the candidate results for the time period of x in each loop, the accuracy of the output recognition results can be further improved while ensuring real-time performance.

[0120] When x is less than i and x is less than j, since the i+y×j-th candidate result is obtained using the candidate results in the previous i+y×j-1 time periods, after outputting x candidate results in the i+y×j-th time period, when pruning the other candidate results in the x time periods corresponding to the already output x candidate results, it will result in fewer than m candidate results in the i+y×j-th time period. Therefore, in the next time period after outputting x candidate results, z candidate results are selected. At this time, the z i+y×j-1-th candidate results are all obtained using the already output y×x candidate results.

[0121] For example, such as Figure 2 As shown, x = 3. At this time, the optimal result of the 5th estimation is F2. The first 3 candidate results indicated by F2 are B1, C1 and D2 in sequence. After outputting B1, C1 and D2, the other results in the first 3 time periods except for the candidate results that have been output are pruned. At this time, there are two 5th candidate results remaining in the 5th time period. Therefore, when obtaining the 6th candidate result in the 6th time period, only the two 5th candidate results are selected. At this time, z is 2.

[0122] When x is less than the maximum value of j, as the number of iterations increases, the number of target time periods that have been estimated to have the best results but have not yet output recognition results will increase, and the number of unrecognized characters accumulated in electronic devices will also increase, greatly occupying the storage space of electronic devices.

[0123] Optionally, after obtaining and outputting the x candidate results corresponding to the first x time periods used to determine the optimal estimated result of the i-th time, the method further includes: determining the target time period for which the optimal estimated result has been determined but the recognition result has not been output; and adjusting the preset output conditions to output at least one recognition result within the target time period if the target time period meets the preset adjustment conditions.

[0124] The preset adjustment conditions include: the number of characters in the estimated optimal result indication corresponding to the target time period is greater than or equal to the first character number threshold, or the duration of the target time period is greater than or equal to the duration threshold.

[0125] Among them, the first character count threshold is greater than the second character count threshold.

[0126] Optionally, if the preset output condition is that the time period number of the current time period is a multiple of the preset value, and if the target time period meets the preset adjustment condition, the preset output condition is adjusted to output at least one recognition result within the target time period, including: obtaining the remainder between the time period number of the current time period and the preset value, and updating the preset value to the remainder.

[0127] If there are too many target time periods for which the estimated optimal result has been determined but no recognition result has been output, the preset value is updated to the remainder. This can output candidate results for x time periods in the current time period, reducing the number of target time periods for which the estimated optimal result has been determined but no recognition result has been output, and alleviating the storage pressure on electronic devices.

[0128] Below, with Figure 3 For example, a specific instance will be used to illustrate the long speech recognition algorithm. In this embodiment, the audio acquisition device is a video conferencing camera, and the preset output condition is that the current time period number is a multiple of a preset value.

[0129] Step 301: Acquire long audio data collected by the video conferencing camera, use the threshold-based VAD algorithm to determine the starting point of the long audio data, and divide the long audio data into k sub-audio data according to the starting point.

[0130] Step 302: Input the sub-speech data of the i-th time period into the end-to-end recognition model of the encoder-decoder architecture, and extract the fbank features through the filtering (FilterBank, fbank) algorithm.

[0131] In other embodiments, the MFCC features of the sub-speech data can also be extracted using the Mel Frequency Cepstrum Coefficient (MFCC) algorithm.

[0132] Step 303: Input the fbank features into the Encoder to obtain a fixed-dimensional vector.

[0133] Step 304, when i=1, sum the vector output by the Encoder. <bos>Input the decoder to obtain n first recognition results; when i>1, input the vector output by the encoder and the output of the decoder from the previous time period into the decoder to obtain n i-th recognition results.

[0134] Step 305: When i = 1, determine the first m first candidate results from the n first recognition results; when i > 1, input the n i-th recognition results into the Beam search algorithm, combine the m (i-1)-th candidate results and the n i-th recognition results respectively to obtain n×m i-th combined results; determine the first m i-th candidate results from the n×m i-th combined results; determine the i-th estimated optimal result from the first m i-th candidate results.

[0135] In Beam Search, m is equivalent to beam width, which can be input by the user or obtained from other devices. This embodiment does not limit the implementation method of beam width.

[0136] Step 306: If the estimated optimal result of the i-th time period satisfies the preset output conditions, obtain and output the x candidate results corresponding to the first x time periods used to determine the estimated optimal result of the i-th time period.

[0137] Step 307: Determine whether the i-th time period is the last time period and whether the video conferencing camera is still working. If the i-th time period is the last time period and the video conferencing camera is not working, proceed to step 308; if the i-th time period is not the last time period or the video conferencing camera is still working, proceed to step 302.

[0138] Step 308: Determine the target time period for which the estimated optimal result has been determined and no candidate result has been output, output the candidate result of the target time period indicated by the estimated optimal result of the current time period, and end the process.

[0139] The long speech recognition algorithm provided in this embodiment acquires long speech data collected by an audio acquisition device; identifies sub-speech data within a first time segment of the long speech data to obtain n first recognition results; determines the first m first candidate results from the n first recognition results; identifies sub-speech data within the i-th time segment of the long speech data to obtain n i-th recognition results; combines the m (i-1)-th candidate results and the n i-th recognition results to obtain n×m i-th combined results; determines the first m i-th candidate results from the n×m i-th combined results; determines the i-th estimated optimal result from the first m i-th candidate results; and, if the i-th estimated optimal result satisfies a preset output condition, obtains... The algorithm determines and outputs the x candidate results corresponding to the first x time periods when estimating the optimal result for the i-th time period. It then uses these x candidate results to determine the optimal estimated results for each subsequent time period in the long speech data, thus obtaining the recognition result for the long speech data. This approach addresses the problem that the output optimal result is only a local optimal result for the current time period, not the final recognition result for the entire long speech data, which leads to inconsistent real-time recognition results and affects post-processing. Since the x candidate results corresponding to the first x time periods tend to stabilize as i increases, outputting the x candidate results corresponding to the first x time periods each time improves the consistency of the recognition results output each time.

[0140] Meanwhile, since i is less than k, the recognition results for x time segments can be output before the long speech data ends, thus improving real-time performance.

[0141] In addition, if the target time period for which the estimated optimal result has been determined but no recognition result has been output meets the preset adjustment conditions, the preset output conditions can be adjusted to output at least one recognition result within the target time period. This can reduce the number of unrecognized target time periods in electronic devices, reduce the occupation of storage resources, and improve flexibility.

[0142] In addition, since the number of characters contained in each time period is different, when using the preset value for the time period to output the recognition result, the time period with more characters will occupy too much storage resources and affect the running speed of the device, while the time period with fewer characters will affect the recognition accuracy. Therefore, using the number of characters as the preset output condition can ensure the accuracy and improve the running speed of electronic devices.

[0143] In addition, since the Lth character to be output may be located at the beginning or middle of the candidate results, outputting only the first L characters may result in the loss of the remaining character data of the candidate results corresponding to the Lth character. Therefore, using the Lth character to determine the x candidate results corresponding to the first x time periods respectively, and outputting x candidate results can avoid data loss and improve the recognition effect.

[0144] Figure 4 This is a block diagram of an apparatus for a long-term recognition method provided in one embodiment of this application. The apparatus includes at least the following modules: a voice acquisition module 410, a first recognition module 420, a first candidate module 430, a second recognition module 440, a result combination module 450, a second candidate module 460, an optimal determination module 470, a result output module 480, and a result acquisition module 490.

[0145] The voice acquisition module 410 is used to acquire long voice data collected by the audio acquisition device;

[0146] The first recognition module 420 is used to recognize the sub-speech data within the first time period in the long speech data to obtain n first recognition results;

[0147] The first candidate module 430 is used to determine the first m first candidate results from n first recognition results;

[0148] The second recognition module 440 is used to recognize the sub-speech data in the i-th time period of the long speech data and obtain n i-th recognition results;

[0149] The result combination module 450 is used to combine the m (i-1)th candidate results and the n ith recognition results respectively to obtain n×m ith combination results;

[0150] The second candidate module 460 determines the first m i-th candidate results from the n×m i-th combination results;

[0151] The optimal determination module 470 is used to determine the i-th estimated optimal result among the first m i-th candidate results;

[0152] The result output module 480 is used to obtain and output the x candidate results corresponding to the first x time periods used to determine the i-th estimated optimal result when the i-th estimated optimal result meets the preset output conditions.

[0153] The result acquisition module 490 is used to determine the optimal estimation result for each subsequent time period in the long speech data using the x candidate results corresponding to the first x time periods, and to obtain the recognition result of the long speech data.

[0154] For relevant details, please refer to the above method implementation examples.

[0155] It should be noted that the apparatus for long speech recognition provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the long speech recognition method apparatus can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the apparatus for long speech recognition provided in the above embodiments and the long speech recognition method embodiments belong to the same concept, and its specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0156] Figure 5 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 501 and a memory 502.

[0157] Processor 501 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0158] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 502 are used to store at least one instruction, which is executed by the processor 501 to implement the long speech recognition method provided in the method embodiments of this application.

[0159] In some embodiments, the electronic device may also optionally include: a peripheral device interface and at least one peripheral device. The processor 501, memory 502, and peripheral device interface can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface via a bus, signal line, or circuit board. Indicatively, peripheral devices include, but are not limited to: radio frequency circuitry, a touch display screen, audio circuitry, and a power supply.

[0160] Of course, electronic devices may also include fewer or more components, and this embodiment does not limit this.

[0161] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the long speech recognition method of the above-described method embodiments.

[0162] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the long speech recognition method of the above-described method embodiments.

[0163] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.< / bos> < / bos>

Claims

1. A long speech recognition method, characterized in that, The method includes: Acquire long speech data collected by an audio acquisition device, wherein the long speech data includes sub-speech data in k time periods; The sub-speech data within the first time segment of the long speech data is identified to obtain n first identification results; where n is a positive integer. Determine the first m first candidate results from n first recognition results; where m is a positive integer less than or equal to n; The sub-speech data within the i-th time period of the long speech data is identified to obtain n i-th identification results; where i is a positive integer taking values ​​from 2, and the value of i is less than k; The m (i-1)th candidate results and the n ith recognition results are combined to obtain n×m ith combined results; Determine the first m i-th candidate results from the n×m i-th combination results; Determine the i-th estimated optimal result from the first m i-th candidate results; If the estimated optimal result of the i-th time period satisfies the preset output conditions, obtain and output the x candidate results corresponding to the first x time periods used to determine the estimated optimal result of the i-th time period, where x is a positive integer less than or equal to i. The optimal estimation result for each subsequent time period in the long speech data is determined by using the x candidate results corresponding to the first x time periods, and the recognition result of the long speech data is obtained. The preset output conditions include: The current time period number is a multiple of a preset value; or, The number of characters in the target time period for which the estimated optimal result has been determined but the recognition result has not been output is greater than or equal to the second character number threshold.

2. The method according to claim 1, characterized in that, The step of determining the optimal estimation result for each subsequent time period in the long speech data using the x candidate results corresponding to the first x time periods to obtain the recognition result of the long speech data includes: For each value of y, the following steps are executed sequentially until i + y × j is greater than k, where y is a positive integer starting from 1: The sub-speech data within the (i+y×j)th time period of the long speech data is identified to obtain n (i+y×j)th recognition results; j is a positive integer starting from 1, and i+y×j is less than k. When j equals 1, the z-th (i+y×j-1)th candidate result and the n-th i-th recognition result are combined to obtain the n×z-th (i+y×j)th combined result; where z is a positive integer less than or equal to m, and the first y×x candidate results that have been output are used when determining the z-th (i+y×j-1)th candidate result; Determine the first m (i+y×j)th candidate results from the n×z (i+y×j)th combination results; When j is not equal to 1, the m i+y×j-1 candidate results and the n i-th recognition results are combined respectively to obtain n×m i+y×j combined results. Determine the first m candidate results of the (i+y×j)th rank among the n×m combined results; Determine the optimal estimated result for the (i+y×j)th rank among the first m candidate results; If the optimal estimated result of the (i+y×j)th time period satisfies the preset output conditions, starting from the (y×x+1)th candidate result, obtain and output the x candidate results used when determining the optimal estimated result of the (i+y×j)th time period, and obtain the recognition results corresponding to the (y+1)×xth time period.

3. The method according to claim 1, characterized in that, After obtaining and outputting the x candidate results corresponding to the first x time periods used to determine the optimal estimate of the i-th time, the process further includes: Identify the target time period for which the optimal estimation result has been determined but the identification result has not been output; If the target time period meets the preset adjustment conditions, the preset output conditions are adjusted to output at least one recognition result within the target time period.

4. The method according to claim 3, characterized in that, The preset adjustment conditions include: The number of characters indicated by the estimated optimal result corresponding to the target time period is greater than or equal to the first character number threshold; or, The duration of the target time period is greater than or equal to the duration threshold.

5. The method according to claim 1, characterized in that, The preset output condition is that the number of characters corresponding to the target time period is greater than or equal to the second character number threshold. When the i-th estimated optimal result satisfies the preset output condition, the process of obtaining x candidate results corresponding to the first x time periods used to determine the i-th estimated optimal result includes: Obtain the first i candidate results used when determining the optimal result of the i-th estimate; The first L characters are determined from the first i candidate results; Based on the Lth character, determine the x candidate results corresponding to the first x time periods respectively.

6. The method according to claim 5, characterized in that, Based on the Lth character, determine x candidate results corresponding to the first x time periods, including: Determine the x-th candidate result to which the L-th character belongs; The xth candidate result and the previous x-1 candidate results are used to determine the x candidate results corresponding to the previous x time periods.

7. The method according to claim 1, characterized in that, The acquisition of long speech data collected by the audio acquisition device includes: The starting endpoint of the long speech data is determined using a speech endpoint detection algorithm. Based on the starting endpoint and the preset time interval, the sub-speech data for the k time periods are determined.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory connected to the processor, the memory storing a program, which the processor executes to implement the long speech recognition method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement the long speech recognition method as described in any one of claims 1 to 7.

Images