A method for selecting among multiple sets of speech recognition results using confidence scores

Abstract

The present invention relates to the technical field of audio recognition, and specifically to a method for selecting from multiple sets of speech recognition results using confidence scores, comprising: collecting speech audio of a person in a car; sending the speech audio to an offline recognition engine and an online recognition engine respectively, and immediately sending the analysis result of each engine to a scoring model after receiving it, to obtain a corresponding confidence score; judging whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; if the confidence score first output is greater than or equal to the first preset threshold, outputting the analysis result corresponding to the confidence score first output as a recognition result; if the confidence score first output is less than the first preset threshold, waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine. The present invention can solve the defect that the prior art cannot take into account both speed and accuracy when recognizing speech.

Description

Technical Field

[0001] The present invention relates to the technical field of audio recognition, and in particular to a method for selecting among multiple sets of speech recognition results using confidence scores. Background Art

[0002] Due to the limitations of the computing power of the vehicle-side chip and the stability of the network connection, the in-vehicle voice assistant usually deploys the recognition engine on both the vehicle side and the cloud. In the actual use process, it is necessary to select the recognition engine. The general practice is to make judgments based on network timeouts or based on the text recognized by the recognition engine. Judgment based on network timeout means that when the network status is good and the online recognition result is returned before the set timeout, the online recognition result is given priority, otherwise the offline recognition result is used. Judging based on the text recognized by the recognition engine means training semantic models or classifying the recognized text based on template rules, giving priority to instructions that the voice assistant can execute correctly. The problems with the above methods are:

[0003] On the one hand, in the network timeout-based method, since the network conditions of the vehicle system are usually unstable, in order to smoothly interact with the cloud system, it is usually necessary to set a longer waiting network timeout. This results in users usually having to wait for a long time to receive feedback from the vehicle voice assistant. This method ignores the advantage of offline recognition in specific fields, which can quickly and accurately recognize some commands (such as vehicle control commands), resulting in slower feedback speed.

[0004] On the other hand, in the method based on the text recognized by the recognition engine, only the semantic information of the recognition result is considered, ignoring the differences in the acoustic characteristics of the user's speech in multiple scenarios in the car (such as talking to the robot, talking to other people in the car, talking to oneself, etc.). This difference may lead to differences in speech recognition results, resulting in a large deviation between the selection results and the actual words spoken by the user, resulting in inaccurate recognition results. Summary of the invention

[0005] Therefore, the present invention aims to solve the defect that the prior art cannot take into account both speed and accuracy when recognizing speech, and thus provides a method for selecting among multiple sets of speech recognition results using confidence scores.

[0006] According to a first aspect of the present invention, there is provided a method for selecting among multiple sets of speech recognition results using confidence scores, comprising the following steps: collecting speech audio of a person in a vehicle; sending the speech audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine that performs speech recognition using a model deployed on a vehicle; and the online recognition engine is an engine that performs speech recognition using a model deployed on a cloud; immediately sending the analysis result of each engine to a scoring model after receiving it to obtain a corresponding confidence score, wherein each analysis result corresponds to a confidence score, and a higher confidence score indicates a more accurate analysis result; determining whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; if the confidence score first output is greater than or equal to the first preset threshold, outputting the analysis result corresponding to the confidence score first output as a recognition result; if the confidence score first output is less than the first preset threshold, waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine; and determining the feedback result of the speech audio by comprehensively considering the confidence score corresponding to each analysis result.

[0007] Optionally, the confidence score corresponding to each analysis result is integrated to determine the feedback result of the speech audio, including: judging whether the confidence score corresponding to the analysis result of each engine is greater than or equal to a second preset threshold, and counting the number of confidence scores greater than or equal to the second preset threshold; if the number of confidence scores greater than or equal to the second preset threshold is 0, sending a request to re-collect the speech audio; if the number of confidence scores greater than or equal to the second preset threshold is 1, outputting the analysis result corresponding to the confidence score as the recognition result; if there are multiple confidence scores greater than or equal to the second preset threshold, using a preset strategy to select the analysis result corresponding to the corresponding confidence score to output as the recognition result.

[0008] Optionally, waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine includes: if the scoring model has not received the analysis result of the target engine after waiting for a preset time, stopping waiting for the scoring model to output the confidence score.

[0009] Optionally, the offline recognition engine includes an offline acoustic model, an offline language model and an offline decoder, and the online recognition engine includes an online acoustic model, an online speech model and an online decoder, wherein the sending of the speech audio to the offline recognition engine and the online recognition engine respectively also includes: sending the speech audio to the offline acoustic model and outputting an offline acoustic probability; obtaining an offline language probability based on the offline acoustic probability output; using the offline decoder in combination with the offline acoustic probability and the language model to decode the speech audio to obtain an analysis result of the offline recognition engine; sending the speech audio to the online acoustic model and outputting an online acoustic probability; obtaining an online language probability based on the online acoustic probability output; using the online decoder in combination with the online language probability to decode the speech audio to obtain an analysis result of the online recognition engine.

[0010] Optionally, the analysis result includes recognition result, decoding cost score and number of audio frames, wherein the higher the decoding cost score, the lower the corresponding confidence score; the recognition result is the language and text recognized by the speech audio, and the lower the matching degree between the number of characters in the language and text and the number of audio frames, the lower the corresponding confidence score.

[0011] Optionally, the decoding cost score includes an acoustic cost score and a language cost score, the acoustic cost score is the negative logarithm of the acoustic probability, and the language cost score is the negative logarithm of the language probability.

[0012] Optionally, the online recognition engine includes an online vehicle-mounted recognition engine and an online general recognition engine, the language model of the online vehicle-mounted recognition engine is trained by corpus of vehicle-mounted scenarios, and the language model of the online general recognition engine is trained by corpus of general scenarios.

[0013] According to a second aspect of the present invention, there is provided a system for selecting among multiple sets of speech recognition results using confidence scores, comprising: an acquisition module for acquiring speech audio of a person in a vehicle; a sending module for sending the speech audio to an offline recognition engine and an online recognition engine, respectively, wherein the offline recognition engine is an engine that uses a model deployed on a vehicle to perform speech recognition; and the online recognition engine is an engine that uses a model deployed on a cloud to perform speech recognition; a scoring module for sending the analysis result of each engine to a scoring model immediately after receiving it to obtain a corresponding confidence score, wherein each analysis result corresponds to a confidence score, and a higher confidence score indicates a more accurate analysis result; a judgment module for judging whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; and a determination module for determining a feedback result of the speech audio by comprehensively considering the confidence score corresponding to each analysis result.

[0014] According to a third aspect of the present invention, there is provided a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-mentioned method of selecting among multiple sets of speech recognition results using confidence scores by executing the computer instructions.

[0015] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing the computer to execute the above-mentioned method of selecting among multiple sets of speech recognition results using confidence scores.

[0016] The technical solution of the present invention has the following advantages:

[0017] 1. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, comprising the following steps: collecting speech audio of a person in a vehicle; sending the speech audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine that performs speech recognition using a model deployed on a vehicle; and the online recognition engine is an engine that performs speech recognition using a model deployed on a cloud; immediately sending the analysis result of each engine to a scoring model after receiving it to obtain a corresponding confidence score, wherein each analysis result corresponds to a confidence score, and a higher confidence score indicates a more accurate analysis result; determining whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; if the confidence score first output is greater than or equal to the first preset threshold, outputting the analysis result corresponding to the confidence score first output as a recognition result; if the confidence score first output is less than the first preset threshold, waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine; and determining the feedback result of the speech audio by comprehensively considering the confidence score corresponding to each analysis result. Through the above settings, the voice audio of the people in the car is first collected, and then the collected voice audio is sent to the offline recognition engine and the online recognition engine. After each recognition engine obtains the analysis result, it is immediately sent to the scoring model. The scoring model scores each analysis result to obtain a confidence score. If the confidence score output first is greater than or equal to the first preset threshold, the analysis result corresponding to the confidence score output first is output as the recognition result. Otherwise, the scoring model is waited to output the confidence score corresponding to the analysis result of each engine, and then the confidence score corresponding to each analysis result is comprehensively determined to determine the feedback result of the voice audio, thereby taking into account both the speed and accuracy of voice recognition.

[0018] 2. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the confidence scores corresponding to each analysis result are integrated to determine the feedback result of the speech audio, including: determining whether the confidence score corresponding to the analysis result of each engine is greater than or equal to a second preset threshold, and counting the number of confidence scores greater than or equal to the second preset threshold; if the number of confidence scores greater than or equal to the second preset threshold is 0, sending a request to re-collect the speech audio; if the number of confidence scores greater than or equal to the second preset threshold is 1, outputting the analysis result corresponding to the confidence score as the recognition result; if there are multiple confidence scores greater than or equal to the second preset threshold, using a preset strategy to select the analysis result corresponding to the corresponding confidence score as the recognition result to be output. Through the above settings, if no confidence score is greater than or equal to the first preset threshold, it is determined whether the confidence score corresponding to the analysis result of each engine is greater than or equal to the second preset threshold, and the number of confidence scores greater than or equal to the second preset threshold is counted. If the number of confidence scores greater than or equal to the second preset threshold is 0, a request for re-collecting voice audio is sent; if the number of confidence scores greater than or equal to the second preset threshold is 1, the analysis result corresponding to the confidence score is output as the recognition result; if there are multiple confidence scores greater than or equal to the second preset threshold, the preset strategy is used to select the analysis result corresponding to the corresponding confidence score as the recognition result output, thereby ensuring the accuracy of speech recognition.

[0019] 3. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the step of waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine includes: if the scoring model has not received the analysis result of the target engine after waiting for a preset time, then the step of waiting for the scoring model to output the confidence score is stopped. With the above setting, when the voice audio of the person in the car cannot be recognized, the speech recognition process is exited, thereby preventing the system from entering an infinite loop.

[0020] 4. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the offline recognition engine includes an offline acoustic model, an offline language model and an offline decoder, and the online recognition engine includes an online acoustic model, an online speech model and an online decoder, wherein the sending of the speech audio to the offline recognition engine and the online recognition engine respectively also includes: sending the speech audio to the offline acoustic model and outputting an offline acoustic probability; obtaining an offline language probability according to the offline acoustic probability output; using the offline decoder in combination with the offline acoustic probability and the language model to decode the speech audio to obtain the analysis result of the offline recognition engine; sending the speech audio to the online acoustic model and outputting an online acoustic probability; obtaining an online language probability according to the online acoustic probability output; using the online decoder in combination with the online language probability to decode the speech audio to obtain the analysis result of the online recognition engine. Through the above settings, the offline recognition engine obtains the offline acoustic probability through the offline acoustic model, obtains the offline language probability through the offline language model, and then uses the offline decoder to decode the speech audio to obtain the analysis result of the offline recognition engine; the online recognition engine obtains the online acoustic probability through the online acoustic model, obtains the online language probability through the online language model, and then uses the online decoder to decode the speech audio to obtain the analysis result of the online recognition engine.

[0021] 5. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the analysis results include recognition results, decoding cost scores, and audio frame numbers, wherein the higher the decoding cost score, the lower the corresponding confidence score, and the recognition result is the language and text recognized by the speech audio, and the lower the matching degree between the number of words in the language and text and the number of audio frames, the lower the corresponding confidence score. Through the above settings, two other outputs are added in addition to the recognition results: decoding cost scores and audio frame numbers, thereby making the judgment of confidence scores more accurate.

[0022] 6. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the decoding cost score includes an acoustic cost score and a language cost score, wherein the acoustic cost score is the negative logarithm of the acoustic probability, and the language cost score is the negative logarithm of the language probability. Through the above arrangement, the acoustic cost score can be conveniently calculated from the acoustic probability, and the language cost score can be conveniently calculated from the language probability.

[0023] 7. The present invention provides a method for selecting from multiple sets of speech recognition results using confidence scores, wherein the online recognition engine includes an online vehicle-mounted recognition engine and an online general recognition engine, wherein the language model of the online vehicle-mounted recognition engine is trained with corpus of vehicle-mounted scenes, and the language model of the online general recognition engine is trained with corpus of general scenes. Through the above settings, the online recognition engine has a wider coverage, thereby being able to more accurately recognize the voice audio of people in the car.

[0024] 8. The present invention provides a system for selecting among multiple sets of speech recognition results using confidence scores, comprising: a collection module for collecting speech audio of people in a vehicle; a sending module for sending the speech audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine for speech recognition using a model deployed on a vehicle; and the online recognition engine is an engine for speech recognition using a model deployed on the cloud; a scoring module for sending the analysis results of each engine to a scoring model immediately after receiving the analysis results to obtain a corresponding confidence score, wherein each analysis result corresponds to a confidence score, and a higher confidence score indicates a more accurate analysis result; a judgment module for judging whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; and a determination module for determining the feedback result of the speech audio by comprehensively considering the confidence scores corresponding to each analysis result. Through the above settings, the voice audio of the people in the car is first collected, and then the collected voice audio is sent to the offline recognition engine and the online recognition engine. After each recognition engine obtains the analysis result, it is immediately sent to the scoring model. The scoring model scores each analysis result to obtain a confidence score. If the confidence score output first is greater than or equal to the first preset threshold, the analysis result corresponding to the confidence score output first is output as the recognition result. Otherwise, the scoring model is waited to output the confidence score corresponding to the analysis result of each engine, and then the confidence score corresponding to each analysis result is comprehensively determined to determine the feedback result of the voice audio, thereby taking into account both the speed and accuracy of voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flowchart of a method for selecting among multiple sets of speech recognition results using confidence scores according to an embodiment of the present application;

[0027] Figure 2This is a structural diagram of step S107 of a method for selecting from multiple sets of speech recognition results using confidence scores in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the structure of a system for selecting among multiple sets of speech recognition results using confidence scores according to an embodiment of the present application;

[0029] Figure 4 A schematic diagram of the structure of a computer device according to an embodiment of the present application.

[0030] Explanation of the accompanying drawings: 1. Collection module; 2. Sending module; 3. Scoring module; 4. Judgment module; 5. Determination module. DETAILED DESCRIPTION

[0031] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0033] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0035] Example 1

[0036] The embodiment of the present application provides a method for selecting from multiple sets of speech recognition results using confidence scores, which is mainly used to recognize the speech audio of people in a car. Compared with the traditional method of selecting from multiple sets of speech recognition results using confidence scores, the method can take into account both the speed and accuracy of speech recognition. Figure 1-2 , the method comprises the following steps:

[0037] Step S101, collecting voice audio of people in the car.

[0038] Step S102, sending the voice audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine for voice recognition using a model deployed on a vehicle; and the online recognition engine is an engine for voice recognition using a model deployed on a cloud.

[0039] In order to improve the speed and accuracy of speech audio recognition, multiple offline recognition engines and online recognition engines can be set.

[0040] Step S103, after receiving the analysis result of each engine, it is immediately sent to the scoring model to obtain the corresponding confidence score, wherein each analysis result corresponds to a confidence score, and the higher the confidence score, the more accurate the analysis result.

[0041] The confidence score model scores the correctness of the recognition result based on the output of the decoder. Generally, the calculation formula for defining the confidence score is:

[0042] score = f(costs, frame_count, query_text);

[0043] The input of formula f is the three outputs of the above decoder, costs is the acoustic and language decoding cost score, frame_count is the number of frames of the input audio, and query_text is the recognition result. Formula f can be obtained by modeling and training a large amount of audio data. The model can be a linear regression model, a decision tree model, or other complex deep learning models. Regardless of the model used, the result obtained by modeling is that the higher the score, the more confident the recognition result is correct, and the lower the score, the more likely the recognition result is incorrect.

[0044] Step S104, determining whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold.

[0045] Among them, under normal circumstances, the recognition speed of the online speech recognition engine is often slower than that of the offline recognition engine due to network fluctuations or network delays. Therefore, the confidence score first output by the scoring model is the confidence score corresponding to the offline recognition engine. If the confidence score is greater than or equal to the first preset threshold, the speed of speech recognition can be improved.

[0046] Step S105: If the confidence score outputted first is greater than or equal to the first preset threshold, the analysis result corresponding to the confidence score outputted first is outputted as the recognition result.

[0047] Step S106: If the confidence score outputted first is less than the first preset threshold, wait for the scoring model to output the confidence score corresponding to the analysis result of each engine.

[0048] Step S107, comprehensively analyzing the confidence score corresponding to each analysis result to determine the feedback result of the speech audio.

[0049] Through the above settings, the voice audio of the people in the car is first collected, and then the collected voice audio is sent to the offline recognition engine and the online recognition engine. After each recognition engine obtains the analysis result, it is immediately sent to the scoring model. The scoring model scores each analysis result to obtain a confidence score. If the confidence score output first is greater than or equal to the first preset threshold, the analysis result corresponding to the confidence score output first is output as the recognition result. At this time, the speed of voice recognition can be improved. Otherwise, wait for the scoring model to output the confidence score corresponding to the analysis result of each engine, and then comprehensively determine the feedback result of the voice audio by combining the confidence score corresponding to each analysis result. At this time, the accuracy of voice recognition can be improved, so that both the speed and accuracy of voice recognition can be taken into account.

[0050] Step S107 also includes the following sub-steps:

[0051] Step S2071, determining whether the confidence score corresponding to the analysis result of each engine is greater than or equal to a second preset threshold, and counting the number of confidence scores greater than or equal to the second preset threshold;

[0052] Step S2072: If the number of confidence scores greater than or equal to the second preset threshold is 0, a request for re-collecting voice audio is sent;

[0053] Step S2073, if the number of confidence scores greater than or equal to the second preset threshold is 1, the analysis result corresponding to the confidence score is output as the recognition result;

[0054] Step S2074: If there are multiple confidence scores that are greater than or equal to the second preset threshold, the preset strategy is used to select the analysis results corresponding to the corresponding confidence scores as the recognition results to be output.

[0055] The preset strategies are as follows:

[0056] The first method is to select the one with the highest confidence score from multiple confidence scores and output the analysis result corresponding to the confidence score as the recognition result;

[0057] The second method is to adopt different selection strategies according to the actual business scenarios. For example, a comprehensive score can be obtained by combining multiple online semantic models such as speech classification, named entity recognition, and semantic rejection models, and the comprehensive scores of general results and special results can be compared to finally decide which result to use.

[0058] Through the above settings, if no confidence score is greater than or equal to the first preset threshold, it is determined whether the confidence score corresponding to the analysis result of each engine is greater than or equal to the second preset threshold, and the number of confidence scores greater than or equal to the second preset threshold is counted. If the number of confidence scores greater than or equal to the second preset threshold is 0, a request for re-collecting voice audio is sent; if the number of confidence scores greater than or equal to the second preset threshold is 1, the analysis result corresponding to the confidence score is output as the recognition result; if there are multiple confidence scores greater than or equal to the second preset threshold, the preset strategy is used to select the analysis result corresponding to the corresponding confidence score as the recognition result output, thereby ensuring the accuracy of speech recognition.

[0059] It should be noted that if the scoring model has not received the analysis result of the target engine after waiting for a preset time, the process stops waiting for the scoring model to output the confidence score.

[0060] Through the above settings, when the voice audio of the person in the car cannot be recognized, the voice recognition process is exited, thereby preventing the system from entering an infinite loop.

[0061] The offline recognition engine includes an offline acoustic model, an offline language model and an offline decoder, and the online recognition engine includes an online acoustic model, an online speech model and an online decoder, wherein the sending of the speech audio to the offline recognition engine and the online recognition engine respectively further includes: sending the speech audio to the offline acoustic model and outputting an offline acoustic probability; obtaining an offline language probability according to the offline acoustic probability output; decoding the speech audio using the offline decoder in combination with the offline acoustic probability and the language model to obtain an analysis result of the offline recognition engine; sending the speech audio to the online acoustic model and outputting an online acoustic probability; obtaining an online language probability according to the online acoustic probability output; decoding the speech audio using the online decoder in combination with the online language probability to obtain an analysis result of the online recognition engine.

[0062] Through the above settings, the offline recognition engine obtains the offline acoustic probability through the offline acoustic model, obtains the offline language probability through the offline language model, and then uses the offline decoder to decode the speech audio to obtain the analysis result of the offline recognition engine; the online recognition engine obtains the online acoustic probability through the online acoustic model, obtains the online language probability through the online language model, and then uses the online decoder to decode the speech audio to obtain the analysis result of the online recognition engine.

[0063] The analysis result includes the recognition result, the decoding cost score and the number of audio frames, wherein the higher the decoding cost score, the lower the corresponding confidence score, and the recognition result is the language and text recognized by the speech audio, the lower the matching degree between the number of words in the language and text and the number of audio frames, the lower the corresponding confidence score. Through the above setting, two other outputs are added in addition to the recognition result: the decoding cost score and the number of audio frames, so that the judgment of the confidence score is more accurate.

[0064] The specific meaning of the matching degree between the number of language characters and the number of audio frames is: for example, if only 3 characters are predicted from a 5-second audio, the corresponding confidence score will not be high, and the recognition result is likely to be incorrect.

[0065] The decoding cost score includes an acoustic cost score and a language cost score, wherein the acoustic cost score is the negative logarithm of the acoustic probability, and the language cost score is the negative logarithm of the language probability. Through the above settings, the acoustic cost score can be conveniently calculated from the acoustic probability, and the language cost score can be conveniently calculated from the language probability.

[0066] Among them, the decoding process of speech recognition is the process of combining acoustic models, language models, and pronunciation dictionaries to find the most likely recognition text results. Taking a typical decoder based on WFST (weighted finite state transducer) as an example, the language model combined with the pronunciation dictionary is represented as a weighted directed graph, and each edge in the graph has an input label (pronunciation), an output label (text), and a weight (the cost of the language model, the negative logarithm of the probability). The decoding process is to jump on this graph based on the possible pronunciation and probability of each frame of audio predicted by the acoustic model. Each jump will have an acoustic cost (the negative logarithm of the probability of the pronunciation) and a language cost (the weight on the edge of the jump). Finally, a path with the minimum total cost will be selected. The output labels on the path are connected to form the recognized text result. The acoustic and language cost scores of each jump on the path are also output as the input of the scoring model.

[0067] The online recognition engine includes an online vehicle-mounted recognition engine and an online general recognition engine. The language model of the online vehicle-mounted recognition engine is trained by the corpus of the vehicle-mounted scene, and the language model of the online general recognition engine is trained by the corpus of the general scene. Through the above settings, the online recognition engine has a wider coverage, so that the voice audio of the person in the car can be recognized more accurately.

[0068] Example 2

[0069] The present invention provides a system for selecting among multiple sets of speech recognition results using confidence scores, which is used to implement the method for selecting among multiple sets of speech recognition results using confidence scores in the above-mentioned embodiment, referring to Figure 3 , including the following modules:

[0070] Collection module 1, used to collect voice and audio of people in the car;

[0071] A sending module 2 is used to send the voice audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine for voice recognition using a model deployed on the vehicle; and the online recognition engine is an engine for voice recognition using a model deployed on the cloud;

[0072] Scoring module 3, used to send the analysis results of each engine to the scoring model immediately after receiving them, to obtain corresponding confidence scores, wherein each analysis result corresponds to a confidence score, and the higher the confidence score, the more accurate the analysis result;

[0073] A judgment module 4 is used to judge whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold;

[0074] The determination module 5 is used to determine the feedback result of the speech audio by comprehensively analyzing the confidence score corresponding to each analysis result.

[0075] Through the above settings, the voice audio of the people in the car is first collected, and then the collected voice audio is sent to the offline recognition engine and the online recognition engine. After each recognition engine obtains the analysis result, it is immediately sent to the scoring model. The scoring model scores each analysis result to obtain a confidence score. If the confidence score output first is greater than or equal to the first preset threshold, the analysis result corresponding to the confidence score output first is output as the recognition result. Otherwise, the scoring model is waited to output the confidence score corresponding to the analysis result of each engine, and then the confidence score corresponding to each analysis result is comprehensively determined to determine the feedback result of the voice audio, thereby taking into account both the speed and accuracy of voice recognition.

[0076] Example 3

[0077] Reference Figure 4An embodiment of the present invention further provides a computer device, which includes a processor and a memory, wherein the processor and the memory may be connected via a bus or other means, and the figure takes the connection via a bus as an example.

[0078] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above chips.

[0079] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor executes various functional applications and data processing of the processor by running the non-volatile software programs, instructions and modules stored in the memory, that is, the method of using confidence points in the above method embodiment to select from multiple sets of speech recognition results is implemented.

[0080] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] The specific details of the above-mentioned computer device can be understood by referring to the corresponding related descriptions and effects in the embodiments shown in Figures 1 to 3, and will not be repeated here.

[0082] The embodiment of the present invention also provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the method of selecting from multiple sets of speech recognition results using confidence scores in any of the above method embodiments. The storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0083] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for selecting among multiple sets of speech recognition results using confidence scores, characterized in that: The steps include: Collect voice audio from people in the car; The speech audio is sent to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine for speech recognition using a model deployed on a vehicle; the online recognition engine is an engine for speech recognition using a model deployed on a cloud, the offline recognition engine includes an offline acoustic model, an offline language model and an offline decoder, and the online recognition engine includes an online acoustic model, an online speech model and an online decoder, wherein the sending of the speech audio to the offline recognition engine and the online recognition engine respectively further includes: sending the speech audio to the offline acoustic model and outputting an offline acoustic probability; obtaining an offline language probability according to the offline acoustic probability output; decoding the speech audio using the offline decoder in combination with the offline acoustic probability and the offline language model to obtain an analysis result of the offline recognition engine; sending the speech audio to the online acoustic model and outputting an online acoustic probability; obtaining an online language probability according to the online acoustic probability output; decoding the speech audio using the online decoder in combination with the online language probability to obtain an analysis result of the online recognition engine; The analysis result includes a recognition result, a decoding cost score, and the number of audio frames, wherein the higher the decoding cost score, the lower the corresponding confidence score; the recognition result is the language and text recognized by the speech audio, and the lower the matching degree between the number of characters in the language and text and the number of audio frames, the lower the corresponding confidence score; The decoding cost score includes an acoustic cost score and a language cost score, wherein the acoustic cost score is the negative logarithm of the acoustic probability, and the language cost score is the negative logarithm of the language probability; After receiving the analysis result of each engine, it is immediately sent to the scoring model to obtain the corresponding confidence score, wherein each analysis result corresponds to a confidence score, and the higher the confidence score, the more accurate the analysis result; Determining whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; If the confidence score outputted first is greater than or equal to the first preset threshold, outputting the analysis result corresponding to the confidence score outputted first as the recognition result; If the confidence score outputted first is less than the first preset threshold, then waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine; The confidence score corresponding to each analysis result is comprehensively determined to determine the feedback result of the speech audio.

2. A method for selecting among multiple sets of speech recognition results using confidence scores according to claim 1, characterized in that: The step of determining the feedback result of the speech audio by synthesizing the confidence score corresponding to each analysis result includes: Determine whether the confidence score corresponding to the analysis result of each engine is greater than or equal to a second preset threshold, and count the number of confidence scores greater than or equal to the second preset threshold; If the number of confidence scores greater than or equal to the second preset threshold is 0, sending a request for re-collecting the voice audio; If the number of confidence scores that is greater than or equal to the second preset threshold is 1, the analysis result corresponding to the confidence score is output as the recognition result; If there are multiple confidence scores that are greater than or equal to the second preset threshold, the preset strategy is used to select the analysis results corresponding to the corresponding confidence scores to be output as the recognition results.

3. A method for selecting among multiple sets of speech recognition results using confidence scores according to claim 1, characterized in that: The step of waiting for the scoring model to output the confidence score corresponding to the analysis result of each engine includes: If the scoring model has not received the analysis result of the target engine after waiting for a preset time, the process stops waiting for the scoring model to output the confidence score.

4. A method for selecting among multiple sets of speech recognition results using confidence scores according to claim 1, characterized in that: The online recognition engine includes an online vehicle-mounted recognition engine and an online general recognition engine. The language model of the online vehicle-mounted recognition engine is trained by corpus of vehicle-mounted scenarios, and the language model of the online general recognition engine is trained by corpus of general scenarios.

5. A system for selecting among multiple sets of speech recognition results using confidence scores, characterized in that: include: The acquisition module is used to collect the voice and audio of people in the car; A sending module, used to send the voice audio to an offline recognition engine and an online recognition engine respectively, wherein the offline recognition engine is an engine for voice recognition using a model deployed on a vehicle; the online recognition engine is an engine for voice recognition using a model deployed on a cloud, the offline recognition engine includes an offline acoustic model, an offline language model and an offline decoder, and the online recognition engine includes an online acoustic model, an online voice model and an online decoder, wherein the sending of the voice audio to the offline recognition engine and the online recognition engine respectively also includes: sending the voice audio to the offline acoustic model and outputting an offline acoustic probability; obtaining an offline language probability according to the offline acoustic probability output; decoding the voice audio using the offline decoder in combination with the offline acoustic probability and the offline language model to obtain an analysis result of the offline recognition engine; sending the voice audio to the online acoustic model and outputting an online acoustic probability; obtaining an online language probability according to the online acoustic probability output; decoding the voice audio using the online decoder in combination with the online language probability to obtain an analysis result of the online recognition engine; The analysis result includes a recognition result, a decoding cost score, and the number of audio frames, wherein the higher the decoding cost score, the lower the corresponding confidence score; the recognition result is the language and text recognized by the speech audio, and the lower the matching degree between the number of characters in the language and text and the number of audio frames, the lower the corresponding confidence score; The decoding cost score includes an acoustic cost score and a language cost score, wherein the acoustic cost score is the negative logarithm of the acoustic probability, and the language cost score is the negative logarithm of the language probability; A scoring module is used to send the analysis results of each engine to the scoring model immediately after receiving them to obtain a corresponding confidence score, wherein each analysis result corresponds to a confidence score, and a higher confidence score indicates a more accurate analysis result; A judgment module, used to judge whether the confidence score first output by the scoring model is greater than or equal to a first preset threshold; The determination module is used to determine the feedback result of the speech audio by comprehensively analyzing the confidence score corresponding to each analysis result.

6. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for selecting among multiple sets of speech recognition results using confidence scores as described in any one of claims 1 to 4 by executing the computer instructions.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions for causing the computer to execute the method for selecting among a plurality of speech recognition results using confidence scores according to any one of claims 1 to 4.

Images