Method and apparatus for responding to voice termination point, medium, program product, and system

Abstract

A method and apparatus for responding to a voice termination point, a medium, a program product and a system. The method for responding to a voice termination point comprises: inputting an acquired audio file into a voice recognition model, to obtain at least one target decoding result output by the voice recognition model, the target decoding result comprising a first text, a probability value of the first text and a corresponding state of the first text in a first weighted finite state transducer, and the first text being a text obtained by the voice recognition model performing voice recognition on the audio file (102); on the basis that the corresponding state of the first text in one or more target decoding results in the first weighted finite state transducer is a termination state, determining a first duration on the basis of the probability value of the first text in the at least one target decoding result (104); and within the first duration after a first moment, canceling a response to a detected voice termination point, wherein the first moment is an acquisition moment of the audio file (106). This method can mitigate the problem of incompleteness of a detected audio file.

Description

Method, device, medium, program product and system for responding to a speech termination point

[0001] The present application claims priority from the Chinese patent application No. 202411261638.0 filed on September 09, 2024, and entitled "Method, device, medium, program product and system for responding to a speech termination point", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of data processing, in particular to a method, device, medium, program product and system for responding to a speech termination point. BACKGROUND

[0003] Voice Activity Detection (VAD) is a technology for identifying speech activity in an audio signal. Its main purpose is to distinguish between speech and non-speech parts, such as silence, noise or background sound, in an audio stream.

[0004] In related technical solutions, a voice interaction system usually includes a voice activity detection module. The voice activity detection module removes non-speech data, so that the subsequent automatic speech recognition module only needs to process speech data, thereby improving processing efficiency and reducing the probability of misrecognition.

[0005] Voice activity detection can detect whether a speech is contained in an audio file, determine a speech start point and a speech termination point. For some interactive scenarios, such as speaking with a long pause due to thinking, voice activity detection may determine that there is a speech termination point due to the pause, resulting in an incomplete audio file obtained by voice activity detection. SUMMARY

[0006] The present application aims to at least solve the problem in the prior art or related art that voice activity detection may determine that there is a speech termination point due to a speaking pause, resulting in an incomplete audio file obtained by voice activity detection.

[0007] To this end, a first aspect of the present application provides a method for responding to a speech termination point.

[0008] A second aspect of the present application provides a device for responding to a speech termination point.

[0009] A third aspect of the present application provides another device for responding to a speech termination point.

[0010] A fourth aspect of the present application provides a readable storage medium.

[0011] A fifth aspect of the present application provides a computer program product.

[0012] A sixth aspect of the present application provides a voice interaction system.

[0013] Therefore, according to a first aspect of the present application, a method for responding to a voice end point is provided, which comprises: inputting a collected audio file into a voice recognition model to obtain at least one target decoding result output by the voice recognition model, the target decoding result comprising a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by performing voice recognition on the audio file by the voice recognition model; determining a first time length according to the probability value of the first text in the at least one target decoding result, based on the state corresponding to the first text in the first weighted finite state transducer being a terminal state; and canceling a response to a detected voice end point within the first time length after a first time, the first time being a collection time of the audio file.

[0014] The present application provides a method for responding to a voice end point. By running the above method for responding to a voice end point, a first time length can be determined according to a collected audio file, and if a voice end point is detected within the first time length after the collection time of the audio file, the response to the detected voice end point is canceled. In this technical solution, within the first time length, the response to the detected voice end point is canceled, so that the voice activation detection can detect a more complete audio file, and the problem that the audio file detected by the voice activation detection is incomplete due to a speaking pause in the related technical solution is improved.

[0015] In the case of detecting a more complete audio file, the user's intention can be more accurately known, thereby improving the accuracy of voice interaction control.

[0016] In addition, the method for responding to a voice end point provided by the present application also has the following additional technical features.

[0017] In some technical solutions, the voice end point comprises: a voice end point given by a neural network-based voice recognition model; and / or a voice end point given by an automatic speech recognition model.

[0018] In this technical solution, the voice end points given by different models are all stopped from responding, thereby improving the accuracy of the voice end points detected by stopping the response.

[0019] In some embodiments, the collected audio file is input into the speech recognition model to obtain at least one target decoding result output by the speech recognition model, specifically including: inputting the collected audio file into the speech recognition model to extract at least one candidate text output by a first conversion layer of the speech recognition model, the first conversion layer being configured to convert the audio file into the candidate text; and decoding each candidate text to obtain at least one target decoding result.

[0020] In this embodiment, the speech recognition model includes a first conversion layer, wherein the first conversion layer is configured to convert the audio file into the candidate text, and after the collected audio file is input into the speech recognition model, the content output by the first conversion layer can be extracted to obtain at least one candidate text recognized from the audio file, i.e., the at least one candidate text. In this process, the collected audio file can be converted into text.

[0021] Each candidate text is decoded to obtain a target decoding result from each candidate text.

[0022] In some embodiments, the prefix beam search decoding is used to decode each candidate text to obtain at least one target decoding result.

[0023] In some embodiments, based on the state corresponding to the first text in the first weighted finite state transducer in the one or more target decoding results being a terminal state, a first duration is determined according to a probability value of the first text in the at least one target decoding result, specifically including: obtaining a first weighted finite state transducer, the first weighted finite state transducer being a weighted finite state transducer constructed using a sample set, the samples in the sample set being texts obtained when an error response voice termination point occurs; comparing a first state attribute with a target state attribute representing a terminal state in the first weighted finite state transducer, the first state attribute being related to the state corresponding to the first text in the first weighted finite state transducer in the at least one target decoding result; based on the first state attribute being the same as the target state attribute, determining that the state corresponding to the first text in the first weighted finite state transducer is a terminal state; and determining the first duration according to the probability value of the first text in the at least one target decoding result.

[0024] In this embodiment, the samples in the sample set are texts obtained when an error response voice termination point occurs. It can be understood that the sample set is a set of samples that are incorrectly identified as having a voice termination point in the voice activity detection process. In other words, each sample in the sample set should not be determined to have a voice termination point, but should be determined to have a voice termination point in the voice activity detection process.

[0025] Based on this, the first weighted finite state transducer constructed using the sample set can represent the possible situation of the first text being wrongly determined as the occurrence of the speech termination point. Based on this, after the target decoding result is identified from the audio file, the state corresponding to the first text in the first weighted finite state transducer is compared with the first weighted finite state transducer, so as to determine whether the first text is the possible situation of being wrongly determined as the occurrence of the speech termination point.

[0026] Specifically, as known from the above, each node in the first weighted finite state transducer has a state attribute. Therefore, after the first text is determined, the state corresponding to the first text in the first weighted finite state transducer can be determined, that is, the state attribute of the corresponding node can be determined. If the first state attribute is the same as the target state attribute representing the termination state in the first weighted finite state transducer, it is considered that the first text is a complete path in the first weighted finite state transducer, that is, the first text is a situation of being wrongly determined as the occurrence of the speech termination point. At this time, the first duration is determined according to the probability value of the first text, so as to stop responding to the detected speech termination point within the first duration.

[0027] In this process, by stopping responding to the detected speech termination point, the speech activation detection can continue, thereby delaying the responding time of the speech termination point, and ultimately making the audio file obtained by the speech activation detection contain longer and more complete speech.

[0028] In addition, in the above technical solution, the first duration is determined according to the probability value of the first text. Compared with using a fixed duration, the speech length of the audio file obtained by the speech activation detection can be dynamically adjusted, and the audio file obtained by the speech activation detection containing a longer blank time is reduced.

[0029] In this process, the accuracy of the audio file obtained by the speech activation detection can be improved, so as to improve the accuracy and recognition efficiency of the speech recognition model.

[0030] In some technical solutions, optionally, the sample set includes a second text and / or a third text; the second text is part of the text extracted from a historical audio file, the historical audio file further includes a fourth text, the fourth text is the text located after the second text, and the time interval between the second text and the fourth text is greater than or equal to a second duration; the same interactive session includes a first sub-audio file and a second sub-audio file having the same prefix, and the third text is the text extracted from the first sub-audio file.

[0031] In this technical solution, the sample set includes three possible situations, that is, the sample set includes only the second text, the sample set includes only the third text, or the sample set includes the second text and the third text.

[0032] In some embodiments, the historical audio file, i.e., a historical collected audio file, is obtained for analysis. When performing speech recognition on the historical audio file, recognized text and a timestamp corresponding to the recognized text are obtained. If a time interval between text at an earlier timestamp (i.e., second text in the present application) and text at a later timestamp (i.e., fourth text in the present application) is greater than or equal to a second time length, it is determined that there is a speaking pause. In this case, a speech termination point should not be determined at the pause. Accordingly, the text at the earlier timestamp (i.e., second text in the present application) is used as a sample in the sample set. The second time length can be set according to actual needs.

[0033] In some embodiments, the second sub-audio file is located after the first sub-audio file, the first sub-audio file is a sub-audio file that is not recognized by the natural language model, and the second sub-audio file is a sub-audio file that is recognized by the natural language model.

[0034] In this embodiment, the first sub-audio file and the second sub-audio file can be understood as voice instructions repeatedly sent by a user to a device in an interactive session. When the first sub-audio file is a sub-audio file that is not recognized by the natural language model and the second sub-audio file is a sub-audio file that is recognized by the natural language model, it is determined that the first sub-audio file is a possible case of false determination of a speech termination point. Accordingly, the text extracted from the first sub-audio file, i.e., third text, is used as a sample in the sample set.

[0035] In this embodiment, the sample set can cover common possible cases, so that the first weighted finite state transducer constructed therefrom can meet the discrimination needs of most scenarios, thereby improving the accuracy of speech activation detection.

[0036] In some embodiments, a determinization algorithm and a minimization algorithm are used to optimize the weighted finite state transducer.

[0037] In this embodiment, the determinization algorithm and the minimization algorithm are used to express the weighted finite state transducer with the least number of states. In this process, the least number of states makes the weighted finite state transducer more compact.

[0038] In some embodiments, the first duration is determined according to the probability value of the first text in the at least one target decoding result, specifically including: determining a maximum probability value in the probability values of the at least one first text; obtaining a maximum delay duration and a minimum delay duration; and determining the first duration based on the maximum probability value, the maximum delay duration and the minimum delay duration.

[0039] In the above technical solution, the maximum probability value in the probability values of the at least one first text is determined, so that the first text with the highest output possibility is selected, and the first duration is determined based on the probability value of the first text with the highest possibility.

[0040] In this process, the first duration can be dynamically determined according to the probability value of the first text, so as to improve the accuracy of stopping responding to the detected voice termination point.

[0041] According to a second aspect of the present application, the present application provides an apparatus for responding to a voice termination point, including: a processing unit configured to input an acquired audio file to a speech recognition model to obtain at least one target decoding result output by the speech recognition model, the target decoding result including a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by performing speech recognition on the audio file by the speech recognition model; a determination unit configured to determine a first duration according to the probability value of the first text in the at least one target decoding result, based on the state corresponding to the first text in the first weighted finite state transducer being a termination state in the one or more target decoding results; and a response unit configured to cancel a response to a detected voice termination point within the first duration after a first time, the first time being an acquisition time of the audio file.

[0042] The present application provides an apparatus for responding to a voice termination point, which can determine a first duration according to an acquired audio file, and cancel a response to a detected voice termination point within the first duration after an acquisition time of the audio file if the voice termination point is detected. In this technical solution, by canceling the response to the detected voice termination point within the first duration, the speech activation detection can detect a more complete audio file, and the problem that the audio file detected by the speech activation detection is incomplete due to a speaking pause in related technical solutions is improved.

[0043] In the case of detecting a more complete audio file, the user's intention can be more accurately known, thereby improving the accuracy of voice interaction control.

[0044] In addition, the apparatus for responding to a voice termination point provided by the present application also has the following additional technical features.

[0045] In some embodiments, the speech termination point comprises: a speech termination point given by a neural network-based speech recognition model; and / or a speech termination point given by an automatic speech recognition model.

[0046] In this embodiment, the speech termination points given by different models can all be stopped, thereby improving the accuracy of the speech termination point detected by the stop response.

[0047] In some embodiments, the processing unit is specifically configured to: input the collected audio file into the speech recognition model to extract at least one candidate text output by a first conversion layer of the speech recognition model, the first conversion layer being configured to convert the audio file into the candidate text; and decode each candidate text to obtain at least one target decoding result.

[0048] In this embodiment, the speech recognition model comprises a first conversion layer, wherein the first conversion layer is configured to convert the audio file into the candidate text. After the collected audio file is input into the speech recognition model, the content output by the first conversion layer can be extracted to obtain the text recognized from the audio file, i.e., at least one candidate text. In this process, the collected audio file can be converted into text.

[0049] Each candidate text is decoded to select a target decoding result from each candidate text.

[0050] In some embodiments, the decoding of each candidate text is performed by using prefix beam search decoding to obtain at least one target decoding result.

[0051] In some embodiments, the determination unit is specifically configured to: obtain a first weighted finite state transducer, the first weighted finite state transducer being a weighted finite state transducer constructed by using a sample set, and the samples in the sample set being texts obtained when the speech termination point is in error; compare a first state attribute with a target state attribute representing a termination state in the first weighted finite state transducer, the first state attribute being related to a state corresponding to a first text in the at least one target decoding result in the first weighted finite state transducer; determine that the state corresponding to the first text in the first weighted finite state transducer is the termination state based on the first state attribute being the same as the target state attribute; and determine a first time length according to a probability value of the first text in the at least one target decoding result.

[0052] In the technical solution, the samples in the sample set are texts obtained when the error response voice end points, and it can be understood that the sample set is a set of samples in which the voice end points are incorrectly identified in the voice activity detection process. In other words, each sample in the sample set should not be determined to have a voice end point, but in the case of being incorrectly determined to have a voice end point in the voice activity detection process.

[0053] Therefore, the first weighted finite state transducer constructed using the sample set can represent the possible case of being incorrectly determined to have a voice end point. Based on this, after the target decoding result is identified from the audio file, the state corresponding to the first text in the first weighted finite state transducer is compared with the first weighted finite state transducer to determine whether the first text is a possible case of being incorrectly determined to have a voice end point.

[0054] Specifically, as known from the above, each node in the first weighted finite state transducer has a state attribute. Therefore, after the first text is determined, the state corresponding to the first text in the first weighted finite state transducer can be determined, that is, the state attribute of the corresponding node can be determined. If the first state attribute is the same as the target state attribute representing the end state in the first weighted finite state transducer, it is considered that the first text is a complete path in the first weighted finite state transducer, that is, the first text is a case of being incorrectly determined to have a voice end point. At this time, the first duration is determined according to the probability value of the first text, so as to stop responding to the detected voice end point within the first duration.

[0055] In this process, by stopping responding to the detected voice end point, the voice activity detection can continue, thereby delaying the responding time of the voice end point, and finally making the audio file obtained by the voice activity detection contain longer and more complete voice.

[0056] In addition, in the above technical solution, the first duration is determined according to the probability value of the first text. Compared with using a fixed duration, the voice length of the audio file obtained by the voice activity detection can be dynamically adjusted, and the audio file obtained by the voice activity detection containing a long blank time can be reduced.

[0057] In this process, the accuracy of the audio file obtained by the voice activity detection can be improved, so as to improve the accuracy and recognition efficiency of the speech recognition model.

[0058] In some embodiments, optionally, the sample set comprises the second text and / or the third text; the second text is part of the text extracted from the historical audio file, the historical audio file further comprises a fourth text, the fourth text is text located after the second text, and a time interval between the second text and the fourth text is greater than or equal to the second time length; the same interactive session comprises the first sub-audio file and the second sub-audio file having the same prefix, and the third text is the text extracted from the first sub-audio file.

[0059] In this embodiment, the sample set comprises three possible cases, i.e., the sample set only comprises the second text, the sample set only comprises the third text, or the sample set comprises the second text and the third text.

[0060] In some embodiments, optionally, the historical audio file is an audio file collected historically, the historical audio file is obtained to analyze the historical audio file. Specifically, when performing speech recognition on the historical audio file, text recognized and a timestamp corresponding to the recognized text are obtained, if it is detected that a time interval between the text at an earlier timestamp (i.e., the second text in the present application) and the text at a later timestamp (i.e., the fourth text in the present application) is greater than or equal to the second time length, it is considered that there is a speaking pause, for the speaking pause, a speech termination point should not be determined at the pause, and based on this, the text at the earlier timestamp (i.e., the second text in the present application) is taken as a sample in the sample set. The second time length can be set according to actual needs.

[0061] In some embodiments, optionally, the second sub-audio file is located after the first sub-audio file, the first sub-audio file is a sub-audio file that is not recognized by the natural language model, and the second sub-audio file is a sub-audio file that is recognized by the natural language model.

[0062] In this embodiment, the first sub-audio file and the second sub-audio file can be understood as voice instructions repeatedly sent by a user to a device in an interactive session, in the case that the first sub-audio file is a sub-audio file that is not recognized by the natural language model and the second sub-audio file is a sub-audio file that is recognized by the natural language model, it is considered that the first sub-audio file is a possible case of being incorrectly determined to have a speech termination point, and therefore, the text extracted from the first sub-audio file, i.e., the third text, is taken as a sample in the sample set.

[0063] In this embodiment, the sample set can cover common possible cases, so that the first weighted finite state transducer constructed can meet the discrimination needs of most scenarios, thereby improving the accuracy of voice activation detection.

[0064] In some embodiments, the weighted finite state transducer is optimized by using a determinization algorithm and a minimization algorithm.

[0065] In this embodiment, the determinization algorithm and the minimization algorithm are used to express the weighted finite state transducer with the least number of states, and in the process, the least number of entries of the states makes the weighted finite state transducer more compact.

[0066] In some embodiments, the determining unit is specifically configured to: determine a maximum probability value from the probability values of the at least one first text; obtain a maximum delay duration and a minimum delay duration; and determine the first duration based on the maximum probability value, the maximum delay duration and the minimum delay duration.

[0067] In the above embodiment, the maximum probability value from the probability values of the at least one first text is determined so as to select the first text with the highest output possibility, and the first duration is determined based on the probability value of the first text with the highest possibility.

[0068] In this process, the first duration can be dynamically determined according to the probability value of the first text, so as to improve the accuracy of stopping responding to the detected voice end point.

[0069] According to a third aspect of the present application, the present application provides a device for responding to a voice end point, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the method for responding to a voice end point according to any one of the above embodiments.

[0070] According to a fourth aspect of the present application, the present application provides a readable storage medium, the readable storage medium storing programs or instructions, and the programs or instructions are executed by the processor to implement the steps of the method for responding to a voice end point according to any one of the above embodiments.

[0071] According to a fifth aspect of the present application, the present application provides a computer program product stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps of the method for responding to a voice end point according to any one of the above embodiments.

[0072] According to a sixth aspect of the present application, the present application provides a voice interaction system, comprising: the device for responding to a voice end point according to any one of the above embodiments; and / or the readable storage medium according to the above; and / or the computer program product according to the above.

[0073] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0074] The above and / or additional aspects and advantages of the present application will become apparent and readily understood by describing in connection with the following detailed description, and appended claims, taken in conjunction with the accompanying drawings, in which:

[0075] FIG. 1 shows a flowchart of a method for responding to a speech termination point in an embodiment of the present application;

[0076] FIG. 2 shows a schematic diagram of an example of a first weighted finite state transducer in an embodiment of the present application;

[0077] FIG. 3 shows a schematic diagram of an audio file in an embodiment of the present application;

[0078] FIG. 4 shows a schematic block diagram of an apparatus for responding to a speech termination point in an embodiment of the present application;

[0079] FIG. 5 shows a schematic block diagram of another apparatus for responding to a speech termination point in an embodiment of the present application. DETAILED DESCRIPTION

[0080] In order to more clearly understand the above aspects, features and advantages of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0081] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, and therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0082] In an embodiment of the present application, as shown in FIG. 1, a method for responding to a speech termination point is provided, comprising:

[0083] Step 102, inputting the collected audio file into a speech recognition model to obtain at least one target decoding result output by the speech recognition model, the target decoding result comprising a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by the speech recognition model performing speech recognition on the audio file;

[0084] Step 104, based on the state corresponding to the first text in the first weighted finite state transducer in the one or more target decoding results being a termination state, determining a first time duration according to the probability value of the first text in the at least one target decoding result;

[0085] Step 106, canceling the response to the detected speech termination point within the first time duration after a first time, the first time being a collection time of the audio file.

[0086] The application provides a method for responding to a speech termination point. By running the method for responding to the speech termination point, a first time length can be determined according to an audio file collected, and within the first time length after the collection time of the audio file, if a speech termination point is detected, the response to the detected speech termination point is cancelled. In this embodiment, within the first time length, the speech activation detection can detect a more complete audio file by cancelling the response to the detected speech termination point, thereby improving the problem that the audio file detected by the speech activation detection is incomplete due to the speaking pause in the related embodiments.

[0087] In the case of detecting a more complete audio file, the user's intention can be more accurately known, thereby improving the accuracy of voice interaction control.

[0088] The audio file can be understood as a file obtained by collecting the user's speaking sound.

[0089] In one of the embodiments, the speech recognition model can be an automatic speech recognition (ASR) model, wherein the target of the automatic speech recognition model is to convert the lexical content in human speech into computer-readable input, such as a key, a binary code or a character sequence.

[0090] In one of the embodiments, the target decoding result is a result obtained by decoding the audio file by the speech recognition model.

[0091] For example, in the case that the user says "I want to watch TV", the collected audio file includes the sound of "I want to watch TV", and the result obtained by decoding the audio file by the speech recognition model is "I want to watch TV".

[0092] In one of the embodiments, the start node and the end node are included in the constructed first weighted finite state transducer, and a plurality of nodes are further included between the start node and the end node, the plurality of nodes are connected with the start node and the end node, the start node and the plurality of nodes, the plurality of nodes and the end node, and the plurality of nodes can be connected by text, and each node has a corresponding state attribute.

[0093] Based on this, the state corresponding to the first text in the first weighted finite state transducer can be understood as whether the first text is a path including the start node and the end node in the first weighted finite state transducer.

[0094] In the above embodiment, the judgment of whether the state corresponding to the first text in the first weighted finite state transducer is a terminal state can indirectly judge whether the first text is a text that needs to determine the first duration and stop responding to the detected speech end point within the first duration after the first time. In this process, the accuracy of the detected speech end point of the stop responding can be improved.

[0095] In the above embodiment, the speech recognition model has a problem of recognition accuracy, so when performing speech recognition on the collected audio file, there is a case that the output first text is inaccurate. Therefore, the probability value of the first text is used to represent the possibility of identifying the first text from the collected audio file.

[0096] In some embodiments, the speech end point corresponds to a speech start point, wherein the speech end point refers to the time when the speech disappears when the audio file is subjected to voice activity detection, and the speech start point can be understood as the time when the speech appears when the audio file is subjected to voice activity detection.

[0097] In some embodiments, the collection time can be understood as the time when the audio file is collected.

[0098] In some embodiments, optionally, the terminal state is the state when the first text is connected to the terminal node.

[0099] In some embodiments, optionally, the speech end point includes: a speech end point given by a neural network-based speech recognition model; and / or a speech end point given by an automatic speech recognition model.

[0100] The speech end point given by the neural network-based speech recognition model is also referred to as NNVAD Endpoint, and the speech end point given by the automatic speech recognition model is also referred to as ASR Endpoint.

[0101] In this embodiment, the speech end points given by different models can all be stopped to respond, thereby improving the accuracy of the detected speech end point of the stop responding.

[0102] In some embodiments, optionally, the collected audio file is input into the speech recognition model to obtain at least one target decoding result output by the speech recognition model, specifically including: inputting the collected audio file into the speech recognition model to extract at least one candidate text output by a first conversion layer of the speech recognition model, the first conversion layer being used to convert the audio file into the candidate text; and decoding each candidate text respectively to obtain at least one target decoding result.

[0103] In this embodiment, the speech recognition model comprises a first conversion layer, wherein the first conversion layer is configured to convert the audio file into candidate text. After inputting the collected audio file into the speech recognition model, the content output by the first conversion layer can be extracted to obtain at least one candidate text, i.e., the text recognized from the audio file. In this process, the collected audio file can be converted into text.

[0104] Each candidate text can be decoded to select a target decoding result from each candidate text.

[0105] In some embodiments, each candidate text can be decoded by prefix beam search decoding to obtain at least one target decoding result.

[0106] The prefix beam search decoding is a beam search-based method. It searches all possible output sequences and uses a scoring function to evaluate the likelihood of each sequence. During the search process, the algorithm keeps the scores of the prefixes and selects the most likely output sequence based on these scores. The score of the prefix is the probability value of the first text in this application.

[0107] The main advantage of the prefix beam search algorithm is that it can handle input and output sequences of different lengths and dynamically adjust the size of the search space during the search process, which makes it more efficient than other search algorithms in handling long sequences.

[0108] In this process, the use of prefix beam search decoding can output the most likely candidate text in the at least one candidate text as the target decoding result, thereby improving the accuracy and recognition efficiency of the speech recognition model.

[0109] In one embodiment, the candidate text can be understood as the text possibly output by the first conversion layer when the speech recognition model recognizes the collected audio file.

[0110] In one embodiment, the first conversion layer is a Connectionist Temporal Classification layer, i.e., a CTC layer.

[0111] In some embodiments, optionally, based on the state corresponding to the first text in the first weighted finite state transducer in the one or more target decoding results being a terminal state, the first duration is determined according to the probability value of the first text in the at least one target decoding result, specifically comprising: obtaining the first weighted finite state transducer, the first weighted finite state transducer being a weighted finite state transducer constructed by using a sample set, the samples in the sample set being texts obtained when the voice end point is wrongly responded; comparing the first state attribute with a target state attribute representing a terminal state in the first weighted finite state transducer, the first state attribute being related to the state corresponding to the first text in the first weighted finite state transducer in the at least one target decoding result; based on the first state attribute being the same as the target state attribute, determining that the state corresponding to the first text in the first weighted finite state transducer is a terminal state; and determining the first duration according to the probability value of the first text in the at least one target decoding result.

[0112] In this embodiment, the samples in the sample set are texts obtained when the voice end point is wrongly responded. It can be understood that the sample set is a set of samples that are wrongly identified as having a voice end point in the voice activity detection process. In other words, each sample in the sample set should not be determined to have a voice end point, but is wrongly determined to have a voice end point in the voice activity detection process.

[0113] Based on this, the first weighted finite state transducer constructed by using the sample set can represent the possible situation of being wrongly determined to have a voice end point. Based on this, after the target decoding result is identified from the audio file, the state corresponding to the first text in the first weighted finite state transducer is compared with the first weighted finite state transducer, so as to determine whether the first text is a possible situation of being wrongly determined to have a voice end point.

[0114] Specifically, as known from the foregoing, each node in the first weighted finite state transducer has a state attribute. Therefore, after the first text is determined, the state corresponding to the first text in the first weighted finite state transducer can be determined, that is, the state attribute of the corresponding node can be determined. If the first state attribute is the same as the target state attribute representing a terminal state in the first weighted finite state transducer, it is considered that the first text is a complete path in the first weighted finite state transducer, that is, the first text is a situation of being wrongly determined to have a voice end point. At this time, the first duration is determined according to the probability value of the first text, so as to stop responding to the detected voice end point within the first duration.

[0115] In this process, the voice activation detection can be continuously performed by stopping the response at the detected voice termination point, and the time of the response voice termination point is delayed, so that the audio file obtained by the voice activation detection contains longer and more complete voice.

[0116] In addition, in the above embodiment, the first duration is determined according to the probability value of the first text, compared with using a fixed duration, the voice length of the audio file obtained by the voice activation detection can be dynamically adjusted, and the audio file obtained by the voice activation detection contains a longer time of blank.

[0117] In this process, the accuracy of the audio file obtained by the voice activation detection can be improved, so as to improve the accuracy and recognition efficiency of the voice recognition model.

[0118] Exemplarily, the state corresponding to the first text in the first weighted finite state transducer is a first state attribute, wherein the first state attribute is a first encoding, and the target state attribute is a second encoding, and if the first encoding is the same as the second encoding, the state corresponding to the first text in the first weighted finite state transducer is a termination state.

[0119] Specifically, the second encoding is 13, if the first state attribute is the first encoding, and the first encoding is also 13, then the state corresponding to the first text in the first weighted finite state transducer is a termination state.

[0120] In one of the embodiments, FIG. 2 shows a schematic diagram of an example of the first weighted finite state transducer in the embodiment of the application, as shown in FIG. 2, the single-layer node 0 is the starting node, and the double-layer node 13 is the ending node, each path from the starting node to the ending node represents a sample, for example, “play a song” corresponds to 0→2→3→9→13; “please play” corresponds to 0→4→10→13.

[0121] The first weighted finite state transducer has the advantage that it can compress the representation of the sample set, and the path can be shared by multiple texts, thereby facilitating decoding.

[0122] In some embodiments, optionally, the sample set includes a second text and / or a third text; wherein the second text is a partial text extracted from a historical audio file, the historical audio file further includes a fourth text, the fourth text is a text located after the second text, and the time interval between the second text and the fourth text is greater than or equal to a second duration; wherein the same interactive session includes a first sub-audio file and a second sub-audio file having the same prefix, and the third text is a text extracted from the first sub-audio file.

[0123] In this embodiment, the sample set includes three possible cases, i.e., the sample set only includes the second text, the sample set only includes the third text, or the sample set includes the second text and the third text.

[0124] In some embodiments, optionally, the historical audio file, i.e., the audio file collected historically, is analyzed by obtaining the historical audio file. Specifically, when the historical audio file is subjected to speech recognition, recognized text and a timestamp corresponding to the recognized text are obtained. If it is detected that the time interval between the text at the earlier timestamp (i.e., the second text in the present application) and the text at the later timestamp (i.e., the fourth text in the present application) is greater than or equal to the second time length, it is considered that there is a speaking pause. For the speaking pause, no speech termination point should be determined at the pause. Based on this, the text at the earlier timestamp (i.e., the second text in the present application) is taken as a sample in the sample set. The second time length can be set according to actual use needs.

[0125] For example, the user controls the air conditioner to set a timing of three hours. At this time, a voice instruction of setting a timing of three hours, i.e., "set timing _____ three hours", is issued, where each "_" represents a time interval of 100 ms. Therefore, the time interval between "set" and "timing" is 100 ms, and the time interval between "timing" and "three" is 500 ms. In the case where the value of the second time length is 300 ms, "set timing" is taken as the second text and as a sample in the sample set.

[0126] The interactive session can be understood as a voice conversation between the user and the device. For example, the interactive session is a conversation generated within a period of time before the user sends a voice instruction to the device and a period of time after the device feeds back information to the user.

[0127] In some embodiments, optionally, the second sub-audio file is located after the first sub-audio file. The first sub-audio file is a sub-audio file that is not recognized by the natural language model, and the second sub-audio file is a sub-audio file that is recognized by the natural language model.

[0128] In this embodiment, the first sub-audio file and the second sub-audio file can be understood as a voice instruction repeatedly sent by the user to the device in the interactive session. In the case where the first sub-audio file is a sub-audio file that is not recognized by the natural language model and the second sub-audio file is a sub-audio file that is recognized by the natural language model, it is considered that the first sub-audio file is a possible case of being incorrectly determined to have a speech termination point. Therefore, the text extracted from the first sub-audio file, i.e., the third text, is taken as a sample in the sample set.

[0129] Exemplarily, the first sub-audio file is "Set_Time_____Three_Hours", and the second sub-audio file is "Set_Time_____Three_Hours". Among them, "Set_Time_____Three_Hours" is not recognized by the natural language model, while "Set_Time_____Three_Hours" is recognized by the natural language model. Then, "Set_Time_____Three_Hours" is used as a sample in the sample set.

[0130] In this embodiment, it can make the sample set cover common possible situations, so that the constructed first weighted finite state transducer can meet the discrimination needs of most scenarios, thereby improving the accuracy of voice activity detection.

[0131] In some embodiments, optionally, the sample set can also be based on the second text and the third text, and be expanded by the large language model and artificial expansion.

[0132] In this embodiment, it can make the sample set contain more samples, and the first weighted finite state transducer constructed based on the sample set meets the determination needs in more scenarios, thereby improving the accuracy of voice activity detection.

[0133] In some embodiments, optionally, the weighted finite state transducer is optimized by using the determinization operation algorithm and / or the minimization operation algorithm.

[0134] In this embodiment, the use of the determinization operation (Determinization) algorithm and the minimization operation (Minimization) algorithm can express the weighted finite state transducer with the least number of states. In this process, the entries of the least number of states make the weighted finite state transducer more compact.

[0135] In some embodiments, optionally, the first duration is determined according to the probability value of the first text in at least one target decoding result, specifically including: determining the maximum probability value among the probability values of at least one first text; obtaining the maximum delay duration and the minimum delay duration; determining the first duration based on the maximum probability value, the maximum delay duration and the minimum delay duration.

[0136] In the above embodiment, by determining the maximum probability value among the probability values of at least one first text, the first text with the highest output possibility is selected, so as to determine the first duration based on the probability value of the first text with the highest possibility.

[0137] In this process, the first duration can be dynamically determined according to the probability value of the first text to improve the accuracy of stopping the voice termination point detected by the response.

[0138] In some embodiments, the first duration is calculated by the following formula: ydt = exp(max(log_p)) x (max_dt - min_dt) + min_dt

[0139] wherein ydt represents the first duration, log_p represents the probability value of the at least one first text, max(log_p) represents the maximum probability value among the probability values of the at least one first text, max_dt represents the maximum delay duration, min_dt represents the minimum delay duration, and exp() is an exponential function with the natural constant e as the base.

[0140] Exemplarily, the probability value of the first text is represented by a logarithmic probability score, and in the case that the audio file is "turn up the temperature", the target decoding results are shown in Table 1:

[0141] Table 1

[0142] Then ydt = exp(max(-0.000575, -10.467508)) x (max_dt - min_dt) + min_dt.

[0143] wherein the maximum delay duration and the minimum delay duration are preset parameters, and specific values thereof are not described herein.

[0144] In one of the embodiments, in the method of responding to the speech endpoint, if the state corresponding to any one of the first texts in the one or more target decoding results in the first weighted finite state transducer is a terminal state, a semantic judgment non-stop signal is generated, wherein the semantic judgment non-stop signal is used to generate the first duration.

[0145] As shown in FIG. 3, when the first text in the target decoding result is "turn the air conditioner to the maximum wind speed", the starting point of the speech is recognized at "turn", and T1 or T2 does not work due to the generation of the semantic judgment non-stop signal.

[0146] wherein T1, i.e. NNVAD Endpoint, is the speech endpoint given by the neural network-based speech recognition model, T2, i.e. ASR Endpoint, is the speech endpoint given by the automatic speech recognition model, and the speech feature is Fbank, wherein Fbank is a common speech feature in ASR, and wherein NNAD refers to neural network-based voice activity detection (Voice Activity Detection, VAD).

[0147] In one of the embodiments, as shown in FIG. 4, the application provides an apparatus 400 for responding to a speech termination point, comprising: a processing unit 402 configured to input a collected audio file into a speech recognition model to obtain at least one target decoding result output by the speech recognition model, the target decoding result comprising a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by the speech recognition model performing speech recognition on the audio file; a determination unit 404 configured to determine a first time length according to the probability value of the first text in the at least one target decoding result based on the state corresponding to the first text in the first weighted finite state transducer being a termination state; and a responding unit 406 configured to cancel the response to the detected speech termination point within the first time length after a first time, the first time being a collection time of the audio file.

[0148] The application provides an apparatus 400 for responding to a speech termination point, which can determine a first time length according to a collected audio file, and cancel the response to the detected speech termination point within the first time length after the collection time of the audio file if a speech termination point is detected. In this embodiment, within the first time length, the response to the detected speech termination point is cancelled, so that the voice activity detection can detect a more complete audio file, and the problem that the audio file detected by the voice activity detection is incomplete due to the speaking pause in the related embodiments is improved.

[0149] In the case of detecting a more complete audio file, the user's intention can be more accurately known, thereby improving the accuracy of voice interaction control.

[0150] The audio file can be understood as a file obtained by collecting the user's speaking voice.

[0151] In one of the embodiments, optionally, the speech recognition model can be an automatic speech recognition (ASR) model, wherein the target of the automatic speech recognition model is to convert the lexical content in human speech into computer-readable input, such as key, binary code or character sequence.

[0152] In one of the embodiments, optionally, the target decoding result is a result obtained by the speech recognition model decoding the audio file.

[0153] For example, in the case that the user says "I want to watch TV", the collected audio file includes the voice of "I want to watch TV", at this time, the result obtained by the speech recognition model decoding the audio file is "I want to watch TV".

[0154] In one of the embodiments, optionally, the start node and the end node are included in the constructed first weighted finite state transducer, and a plurality of nodes are further included between the start node and the end node, the plurality of nodes are connected with the start node and the end node, the start node and the plurality of nodes, the plurality of nodes and the end node, and the plurality of nodes are connected by the text, and each node has a corresponding state attribute.

[0155] Based on this, the state corresponding to the first text in the first weighted finite state transducer can be understood as whether the first text is a path containing the start node and the end node in the first weighted finite state transducer.

[0156] In the above embodiment, judging whether the state corresponding to the first text in the first weighted finite state transducer is the end state can indirectly judge whether the first text is a text that needs to determine the first duration and stop responding to the detected speech end point within the first duration after the first time. In this process, the accuracy of stopping responding to the detected speech end point can be improved.

[0157] In the above embodiment, the speech recognition model has a problem of recognition accuracy, so when performing speech recognition on the collected audio file, there is a case that the output first text is inaccurate. Based on this, the probability value of the first text is used to represent the possibility of identifying the first text from the collected audio file.

[0158] In some embodiments, the speech end point corresponds to the speech start point, wherein the speech end point refers to the time when the speech disappears when the audio file is subjected to the voice activity detection, and the speech start point can be understood as the time when the speech appears when the audio file is subjected to the voice activity detection.

[0159] In some embodiments, the collection time can be understood as the time when the audio file is collected.

[0160] In some embodiments, optionally, the end state is the state when the first text is connected with the end node.

[0161] In some embodiments, optionally, the speech end point includes: a speech end point given by a neural network-based speech recognition model; and / or a speech end point given by an automatic speech recognition model.

[0162] The speech end point given by the neural network-based speech recognition model is also called NNVAD Endpoint, and the speech end point given by the automatic speech recognition model is also called ASR Endpoint.

[0163] In this embodiment, the speech termination points given by different models can all be stopped to respond, thereby improving the accuracy of the speech termination point detected by the stop response.

[0164] In some embodiments, the processing unit 402 is further configured to: input the collected audio file into the speech recognition model to extract at least one candidate text output by a first conversion layer of the speech recognition model, the first conversion layer being configured to convert the audio file into the candidate text; and decode each candidate text to obtain at least one target decoding result.

[0165] In this embodiment, the speech recognition model comprises a first conversion layer, wherein the first conversion layer is configured to convert the audio file into the candidate text. After the collected audio file is input into the speech recognition model, the content output by the first conversion layer can be extracted to obtain at least one candidate text recognized from the audio file, i.e., the text. In this process, the collected audio file can be converted into text. Each candidate text can be decoded to select a target decoding result from each candidate text.

[0166] In some embodiments, the decoding of each candidate text is performed by prefix beam search decoding to obtain at least one target decoding result.

[0167] The prefix beam search decoding is a method based on beam search. It searches all possible output sequences and uses a score function to evaluate the likelihood of each sequence. During the search process, the algorithm retains the scores of the prefixes and selects the most likely output sequence based on these scores. The score of the prefix, i.e., the probability value of the first text in this application.

[0168] The main advantage of the prefix beam search algorithm is that it can handle input and output sequences of different lengths and dynamically adjust the size of the search space during the search process, which makes it more efficient than other search algorithms when dealing with long sequences.

[0169] In this process, the use of prefix beam search decoding can output the most likely candidate text in the at least one candidate text as the target decoding result, thereby improving the accuracy and recognition efficiency of the speech recognition model.

[0170] In one embodiment, the candidate text can be understood as the text that the speech recognition model can output from the first conversion layer when recognizing the collected audio file.

[0171] In one of the embodiments, the first conversion layer is a Connectionist Temporal Classification layer, i.e., a CTC layer.

[0172] In some embodiments, the determining unit 404 is configured to: obtain a first weighted finite state transducer, the first weighted finite state transducer being a weighted finite state transducer constructed by using a sample set, the samples in the sample set being texts obtained when a voice end point is wrongly responded; compare a first state attribute with a target state attribute representing an end state in the first weighted finite state transducer, the first state attribute being related to a state corresponding to the first text in the first weighted finite state transducer in at least one target decoding result; determine that the state corresponding to the first text in the first weighted finite state transducer is an end state based on the first state attribute being the same as the target state attribute; and determine a first time length according to a probability value of the first text in the at least one target decoding result.

[0173] In this embodiment, the samples in the sample set are texts obtained when a voice end point is wrongly responded. It can be understood that the sample set is a set of samples in which a voice end point is wrongly identified as existing in a voice activation detection process. In other words, each sample in the sample set should not be determined as a voice end point, but is wrongly determined as a voice end point in the voice activation detection process.

[0174] Based on this, the first weighted finite state transducer constructed by using the sample set can represent a possible case in which a voice end point is wrongly determined. Based on this, after a target decoding result is identified from an audio file, a state corresponding to a first text in the first weighted finite state transducer in the target decoding result is compared with the first weighted finite state transducer, so as to determine whether the first text is a possible case in which a voice end point is wrongly determined.

[0175] Specifically, as known from the above, each node in the first weighted finite state transducer has a state attribute. Therefore, after the first text is determined, a state corresponding to the first text in the first weighted finite state transducer can be determined, i.e., a state attribute of a corresponding node can be determined. If the first state attribute is the same as a target state attribute representing an end state in the first weighted finite state transducer, it is considered that the first text is a complete path in the first weighted finite state transducer, i.e., the first text is a case in which a voice end point is wrongly determined. At this time, a first time length is determined according to a probability value of the first text, so as to stop responding to a detected voice end point within the first time length.

[0176] In this process, the voice activation detection can be continuously performed by stopping the response at the detected voice end point, the time of the delayed response voice end point is delayed, and finally, the audio file obtained by the voice activation detection contains longer and more complete voice.

[0177] In addition, in the above embodiment, the first duration is determined according to the probability value of the first text, compared with using a fixed duration, the voice length of the audio file obtained by the voice activation detection can be dynamically adjusted, and the audio file obtained by the voice activation detection contains less long blank.

[0178] In this process, the accuracy of the audio file obtained by the voice activation detection can be improved, so as to improve the accuracy and recognition efficiency of the voice recognition model.

[0179] Exemplarily, the state corresponding to the first text in the first weighted finite state transducer is a first state attribute, wherein the first state attribute is a first encoding, and the target state attribute is a second encoding, if the first encoding is the same as the second encoding, the state corresponding to the first text in the first weighted finite state transducer is a terminal state.

[0180] Specifically, the second encoding is 13, if the first state attribute is the first encoding, and the first encoding is also 13, the state corresponding to the first text in the first weighted finite state transducer is a terminal state.

[0181] In some embodiments, optionally, the sample set includes a second text and / or a third text; wherein the second text is a part of text extracted from the historical audio file, the historical audio file further includes a fourth text, the fourth text is a text located after the second text, and the time interval between the second text and the fourth text is greater than or equal to the second duration; wherein the same interactive session includes the first sub-audio file and the second sub-audio file with the same prefix, and the third text is the text extracted from the first sub-audio file. In this embodiment, the sample set includes three possible cases, that is, the sample set only includes the second text, the sample set only includes the third text, or the sample set includes the second text and the third text.

[0182] In some embodiments, optionally, historical audio files, i.e., historically acquired audio files, are obtained for analysis. Specifically, when performing speech recognition on historical audio files, the recognized text and its corresponding timestamp are obtained. If the time interval between the text with the earlier timestamp (i.e., the second text in this application) and the text with the later timestamp (i.e., the fourth text in this application) is greater than or equal to a second duration, a pause in speech is considered to exist. For pauses in speech, a speech termination point should not be determined at the pause. Based on this, the text with the earlier timestamp (i.e., the second text in this application) is used as a sample in the sample set. The second duration can be set according to actual usage needs.

[0183] For example, when a user controls the air conditioner to set a timer for three hours, a voice command to set the timer for three hours will be issued, namely "Set time for three hours". Each "_" represents a time interval of 100ms. Therefore, the time interval between "set" and "time" is 100ms, and the time interval between "time" and "three" is 500ms. If the second duration is 300ms, then "set time" will be used as the second text and as a sample in the sample set.

[0184] An interactive session can be understood as a voice dialogue between a user and a device. For example, an interactive session is a conversation that takes place between a period of time before the user sends a voice command to the device and a period of time after the device provides feedback to the user.

[0185] In some embodiments, optionally, a second sub-audio file is located after a first sub-audio file, wherein the first sub-audio file is a sub-audio file not recognized by the natural language model, and the second sub-audio file is a sub-audio file recognized by the natural language model.

[0186] In this embodiment, the first sub-audio file and the second sub-audio file can be understood as voice commands repeatedly sent by the user to the device during the interactive session. If the first sub-audio file is not recognized by the natural language model and the second sub-audio file is recognized by the natural language model, then the first sub-audio file is considered to be a possible case of incorrectly determining that a voice termination point has occurred. Therefore, the text extracted from the first sub-audio file, that is, the third text, is used as a sample in the sample set.

[0187] Exemplarily, the first sub-audio file is "Set time for three hours", and the second sub-audio file is "Set time for three hours". Among them, "Set time for three hours" is not recognized by the natural language model, while "Set time for three hours" is recognized by the natural language model. Then, "Set time for three hours" is used as a sample in the sample set.

[0188] In this embodiment, it is possible to make the sample set cover common possible situations, so that the constructed first weighted finite state transducer can meet the discrimination needs of most scenarios, thereby improving the accuracy of voice activation detection.

[0189] In some embodiments, optionally, the sample set can also be based on the second text and the third text, and be expanded by the large language model and artificial expansion.

[0190] In this embodiment, it is possible to make the sample set contain more samples, and the first weighted finite state transducer constructed based on the sample set meets the determination needs in more scenarios, thereby improving the accuracy of voice activation detection.

[0191] In some embodiments, optionally, a determinization operation algorithm and / or a minimization operation algorithm are used to optimize the weighted finite state transducer.

[0192] In this embodiment, the use of the use of the determinization operation (Determinization) algorithm and the minimization operation (Minimization) algorithm can express the weighted finite state transducer with the fewest states. In this process, the entries of the fewest states make the weighted finite state transducer more compact.

[0193] In some embodiments, optionally, the determining unit 404 is specifically configured to: determine the maximum probability value among the probability values of at least one first text; obtain the maximum delay duration and the minimum delay duration; and determine the first duration based on the maximum probability value, the maximum delay duration, and the minimum delay duration.

[0194] In the above embodiment, by determining the maximum probability value among the probability values of at least one first text, so as to select the first text with the highest output possibility, and determine the first duration based on the probability value of the first text with the highest possibility.

[0195] In this process, the first duration can be dynamically determined according to the probability value of the first text to improve the accuracy of stopping the detected voice termination point.

[0196] In some embodiments, the first time length is calculated by the following formula: ydt = exp(max(log_p)) x (max_dt-min_dt) + min_dt

[0197] wherein ydt represents the first time length, log_p represents the probability value of the at least one first text, max(log_p) represents the maximum probability value in the probability value of the at least one first text, max_dt represents the maximum delay time length, min_dt represents the minimum delay time length, and exp() is an exponential function with the natural constant e as the base.

[0198] In one of the embodiments, as shown in FIG. 5, the application provides another device 500 for responding to a voice termination point, which comprises a processor 502 and a memory 504, the memory 504 stores programs or instructions executable on the processor 502, and the programs or instructions are executed by the processor to implement the steps of any one of the above methods.

[0199] In this embodiment, the first time length can be determined according to the collected audio file, and within the first time length after the collection time of the audio file, if a voice termination point is detected, the response to the detected voice termination point is cancelled. In this technical solution, within the first time length, the response to the detected voice termination point is cancelled, so that the voice activation detection can detect a more complete audio file, and the problem that the audio file detected by the voice activation detection is incomplete due to the speaking pause in the related technical solution is improved.

[0200] In the case of detecting a more complete audio file, the user's intention can be more accurately known, thereby improving the accuracy of voice interaction control.

[0201] The memory can be used to store software programs and various data. The memory mainly includes a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory can include a volatile memory or a non-volatile memory, or the memory can include both volatile and non-volatile memories. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0202] In some embodiments, optionally, the present application provides a readable storage medium, and the readable storage medium stores programs or instructions, and the programs or instructions are executed by a processor to implement the steps of the method for responding to a voice termination point according to any one of the above.

[0203] In this embodiment, the first time length can be determined according to the collected audio file, and within the first time length after the collection time of the audio file, if a voice termination point is detected, the response to the detected voice termination point is cancelled. In this technical solution, within the first time length, by cancelling the response to the detected voice termination point, the voice activation detection can detect a more complete audio file, and the problem that the audio file detected by the voice activation detection is incomplete due to the existence of a speaking pause in the related technical solution is improved.

[0204] In the case of detecting a more complete audio file, the user's intention can be more accurately understood, thereby improving the accuracy of voice interaction control.

[0205] In some embodiments, optionally, a computer program product is provided, the computer program product is stored in a storage medium, and the computer program product, when executed by at least one processor, implements the steps of the method of responding to a voice termination point according to any one of the above.

[0206] In some embodiments, optionally, the present application provides a voice interaction system, comprising: the device for responding to a voice termination point according to any one of the above; and / or the readable storage medium according to the above; and / or the computer program product according to the above.

[0207] In this embodiment, the voice interaction system can determine a first time length according to the collected audio file, and within the first time length after the collection time of the audio file, if a voice termination point is detected, the response to the detected voice termination point is canceled. In this technical solution, within the first time length, by canceling the response to the detected voice termination point, the voice activation detection can detect a more complete audio file, thereby improving the problem in the related technical solution that the audio file detected by the voice activation detection is incomplete due to the speaking pause.

[0208] In the case of detecting a more complete audio file, the user's intention can be more accurately understood, thereby improving the accuracy of voice interaction control.

[0209] In some embodiments, optionally, the voice interaction system can be deployed in a user device, wherein the user device can be an air conditioner, at this time, the user can use the voice interaction system to voice control the air conditioner to adjust the operating parameters, such as adjusting the cooling temperature, heating temperature, adjusting the wind speed, turning on and off the user selected function, switching between the cooling, heating, dehumidifying and air supply modes.

[0210] In some embodiments, optionally, the user device can be a humidifier, and the user can use the voice interaction system to voice control the humidifier to adjust the operating parameters, such as adjusting the humidity setting value and turning off the humidifier.

[0211] In some embodiments, optionally, the user device can be an air purifier, and the user can use the voice interaction system to voice control the air purifier to adjust the operating parameters, such as adjusting the purification mode of the air purifier.

[0212] In some embodiments, optionally, the user device can be a sweeping robot, and the user can use the voice interaction system to voice control the sweeping robot to perform the cleaning function, perform the cleaning or other selected function.

[0213] The terms "first", "second", etc. in the specification and claims of the present application can explicitly or implicitly include one or more of the features. In the literal description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the front and rear associated objects are in an "or" relationship.

[0214] In the literal description of the present application, it can be understood that, unless there is a clear specification and limitation, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0215] In the claims, the specification and drawings of the present application, the description of the terms "one embodiment", "some embodiments", "a specific embodiment" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the claims, the specification and drawings of the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0216] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of responding to a voice termination point, wherein, The method comprises: inputting the collected audio file into a speech recognition model to obtain at least one target decoding result output by the speech recognition model, the target decoding result comprising a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by performing speech recognition on the audio file by the speech recognition model; based on the state corresponding to the first text in the first weighted finite state transducer being a terminal state in one or more target decoding results, determining a first time length according to the probability value of the first text in the at least one target decoding result; within the first time length after a first time, canceling the response to the detected speech termination point, the first time being the collection time of the audio file.

2. The method of responding to a voice termination point of claim 1, wherein, The speech termination point comprises: a speech termination point given by a neural network-based speech recognition model; and / or a speech termination point given by an automatic speech recognition model.

3. The method of responding to a voice termination point of claim 1 or 2, wherein, The method of inputting the collected audio file into the speech recognition model to obtain the at least one target decoding result specifically comprises: inputting the collected audio file into the speech recognition model to extract at least one candidate text output by a first conversion layer of the speech recognition model, the first conversion layer being used to convert the audio file into the candidate text; decoding each candidate text respectively to obtain at least one target decoding result.

4. The method of responding to a speech termination point according to claim 3, wherein each candidate text is decoded respectively by using prefix beam search decoding to obtain at least one target decoding result.

5. The method of responding to a voice termination point according to any one of claims 1 to 4, wherein, The method of determining a first time length according to the probability value of the first text in the at least one target decoding result based on the state corresponding to the first text in the first weighted finite state transducer being a terminal state specifically comprises: obtaining the first weighted finite state transducer, the first weighted finite state transducer being a weighted finite state transducer constructed by using a sample set, a sample in the sample set being a text obtained when an erroneous speech termination point is responded to; comparing a first state attribute with a target state attribute representing a terminal state in the first weighted finite state transducer, the first state attribute being related to the state corresponding to the first text in the first weighted finite state transducer in the at least one target decoding result; based on the first state attribute being the same as the target state attribute, determining that the state corresponding to the first text in the first weighted finite state transducer is a terminal state; determining a first time length according to the probability value of the first text in the at least one target decoding result.

6. The method of responding to a voice termination point of claim 5, wherein, The sample set comprises a second text and / or a third text; wherein the second text is a partial text extracted from a historical audio file, the historical audio file further comprising a fourth text, the fourth text being a text located after the second text and a time interval between the second text and the fourth text being greater than or equal to a second time length. The same interaction session includes the first sub-audio file and the second sub-audio file with the same prefix, and the third text is the text extracted from the first sub-audio file.

7. The method of responding to a voice termination point of claim 6, wherein, The second sub-audio file is located after the first sub-audio file, the first sub-audio file is a sub-audio file not recognized by a natural language model, and the second sub-audio file is a sub-audio file recognized by the natural language model.

8. The method of responding to a voice termination point of any of claims 5 to 7, wherein, The weighted finite state transducer is optimized by using a determinization operation algorithm and a minimization operation algorithm.

9. The method of responding to a voice termination point of any of claims 5 to 7, wherein, The first duration is determined according to the probability value of the first text in the at least one target decoding result, specifically including: determining a maximum probability value in the probability values of the at least one first text; obtaining a maximum delay duration and a minimum delay duration; determining the first duration based on the maximum probability value, the maximum delay duration, and the minimum delay duration.

10. An apparatus for responding to a voice termination point, wherein, including: a processing unit configured to input an acquired audio file to a speech recognition model to obtain at least one target decoding result output by the speech recognition model, the target decoding result including a first text, a probability value of the first text, and a state corresponding to the first text in a first weighted finite state transducer, the first text being a text obtained by performing speech recognition on the audio file by the speech recognition model; a determining unit configured to determine a first duration according to a probability value of the first text in the at least one target decoding result, based on the state corresponding to the first text in the first weighted finite state transducer being a terminal state in one or more target decoding results; a responding unit configured to cancel a response to a detected speech termination point within the first duration after a first time, the first time being an acquisition time of the audio file.

11. An apparatus for responding to a voice termination point, wherein, including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the method for responding to a speech termination point according to any one of claims 1 to 9.

12. A readable storage medium, wherein, The readable storage medium stores programs or instructions, which are executed by the processor to implement the steps of the method for responding to a speech termination point according to any one of claims 1 to 9.

13. A computer program product, the computer program product being stored in a storage medium, wherein, The computer program product is executed by at least one processor to implement the steps of the method for responding to a speech termination point according to any one of claims 1 to 9.

14. A voice interaction system wherein, including: the device for responding to a speech termination point according to claim 10 or 11; and / or the readable storage medium according to claim 12; and / or the computer program product according to claim 13.

Images