Dialogue round end judgment method and device, electronic equipment, medium and vehicle

Abstract

The invention relates to a dialogue round end judgment method and device, electronic equipment, a medium and a vehicle. The method comprises the steps that a first effective voice segment of a to-be-recognized voice frame is acquired, and first text content corresponding to the first effective voice segment is recognized; inputting the first effective voice segment and the first text content into a preset voice recognition model, and obtaining a first confidence score output by the preset voice recognition model; the first confidence score is used for representing the probability that the user does not speak a sentence according to the first effective voice segment and the first text content; according to the first confidence score, the preset mute duration is adjusted to be the target mute duration; the target mute duration is in positive correlation with the first confidence score; and determining whether the dialogue of the to-be-recognized voice frame is ended based on the target mute duration. By adopting the method, whether the current conversation of the user is finished can be more flexibly judged, and the voice interaction experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, electronic device, medium and vehicle for determining the end of a conversation turn. Background Art

[0002] The interaction in the vehicle cabin is mainly based on voice interaction. The voice interaction function can be realized through the car assistant. After the user says a sentence, whether the car assistant can quickly reply and execute is an important reference for users to judge whether the interaction is friendly. Figure 1 As shown, Figure 1 It is a flow chart of voice interaction in the vehicle cabin in the prior art. The specific voice interaction process is as follows: First, the user wakes up the car assistant through the preset wake-up word to make it enter the working state. For example, the preset wake-up word can be: "XX classmate", "Xiao X", etc. Then, the car assistant detects whether the user has valid voice input through VAD technology. If valid voice is detected, it is sent to the voice recognition module for recognition, otherwise it times out and exits. For example, after waking up the car assistant, the user does not issue a specific instruction within the preset time. The car assistant will ask again, "What help do you need?" If no corresponding instruction is received within a few seconds, it times out and exits. Then, the voice recognition module converts the valid voice into recognized text and sends it to the downstream semantic understanding module. The semantic understanding module obtains the user's intention, action and other information based on the recognition result and sends it to the downstream control unit, and the control unit performs the corresponding action. In VAD (Voice Activity Detection) technology, a judgment result of valid speech and invalid speech (silence) is output for each speech frame. Multiple consecutive frames of valid speech combined together will be judged as the user starting to speak. After that, multiple consecutive frames of silence will be judged as the user has finished speaking.

[0003] In related technologies, in order to reduce user waiting time, the VAD will set a relatively small silence threshold for judging when a user has finished speaking a sentence, usually within 1 second. Although a smaller VAD silence threshold can optimize the response speed of the car assistant, in the actual voice interaction process, the user may hesitate or pause when talking to the car. At this time, if the user's pause time exceeds the VAD silence threshold, the system will judge that the user has finished speaking and enter the subsequent interaction process. This method of setting the silence threshold to a fixed value is not conducive to the car assistant correctly understanding the user's semantics, and thus cannot correctly execute user commands, resulting in a poor user experience. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a method, device, electronic device, medium and vehicle for determining the end of a conversation turn.

[0005] In a first aspect, the present disclosure provides a method for determining the end of a conversation turn, the method comprising:

[0006] Acquire a first valid voice segment of a voice frame to be recognized, and recognize a first text content corresponding to the first valid voice segment;

[0007] Inputting the first valid voice segment and the first text content into a preset voice recognition model, and obtaining a first confidence score output by the preset voice recognition model; the first confidence score is used to indicate the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content;

[0008] According to the first confidence score, adjusting the preset silence duration to a target silence duration; the target silence duration is positively correlated with the first confidence score;

[0009] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0010] As an optional implementation of the embodiment of the present disclosure, the process of presetting the first confidence score output by the speech recognition model includes:

[0011] The preset speech recognition model extracts a first acoustic feature of the first valid speech segment, and extracts a first text feature of the first text content;

[0012] The preset speech recognition model performs feature fusion on the first acoustic feature and the first text feature to obtain a first fusion feature;

[0013] The preset speech recognition model performs confidence classification on the first fusion feature, obtains the first confidence score and outputs it.

[0014] As an optional implementation of the embodiment of the present disclosure, the first acoustic feature includes at least one of the following: a tone feature, a drag feature, and a pronunciation duration; and the first text feature includes a semantic integrity feature.

[0015] As an optional implementation of the embodiment of the present disclosure, obtaining a first valid voice segment of a voice frame to be recognized includes:

[0016] Obtain the speech frame to be recognized;

[0017] Extracting audio feature information of the speech frame to be recognized;

[0018] Audio clustering is performed according to the audio feature information to obtain the first valid voice segment.

[0019] As an optional implementation of the embodiment of the present disclosure, identifying the first text content corresponding to the first valid voice segment includes:

[0020] Preprocessing the first valid speech segment, wherein the preprocessing includes at least one of the following: denoising, voice extraction, and voice enhancement;

[0021] The preprocessed first valid voice segment is subjected to voice recognition and converted into corresponding first text content.

[0022] As an optional implementation of the embodiment of the present disclosure, adjusting the preset silence duration to the target silence duration according to the first confidence score includes:

[0023] When the first confidence score is greater than or equal to a first preset threshold, adjusting the preset silence duration to a first silence duration;

[0024] When the first confidence score is greater than or equal to a second preset threshold and less than the first preset threshold, adjusting the preset silence duration to a second silence duration;

[0025] When the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted.

[0026] As an optional implementation of the embodiment of the present disclosure, the method further includes:

[0027] If a second valid voice segment is detected within the first silence duration, inputting the second valid voice segment into the preset voice recognition model, and obtaining a second confidence score of the second valid voice segment output by the preset voice recognition model;

[0028] According to the second confidence score, adjusting the first remaining silence duration to the target silence duration; the first remaining silence duration is the difference between the first silence duration and the first detection duration, and the first detection duration is the duration between the start time of the first silence duration and the time when the second valid voice segment is detected;

[0029] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0030] In a second aspect, an embodiment of the present disclosure provides a device for determining the end of a conversation turn, the device comprising:

[0031] An acquisition module, used to acquire a first valid voice segment of a voice frame to be recognized, and recognize a first text content corresponding to the first valid voice segment;

[0032] An input module, used to input the first valid voice segment and the first text content into a preset voice recognition model, and obtain a first confidence score output by the preset voice recognition model; the first confidence score is used to represent the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content;

[0033] an adjustment module, configured to adjust a preset silence duration to a target silence duration according to the first confidence score; the target silence duration is positively correlated with the first confidence score;

[0034] The judgment module is used to determine whether the conversation of the speech frame to be recognized is ended based on the target silence duration.

[0035] As an optional implementation of the embodiment of the present disclosure, the process of presetting the first confidence score output by the speech recognition model includes:

[0036] The preset speech recognition model extracts a first acoustic feature of the first valid speech segment, and extracts a first text feature of the first text content;

[0037] The preset speech recognition model performs feature fusion on the first acoustic feature and the first text feature to obtain a first fusion feature;

[0038] The preset speech recognition model performs confidence classification on the first fusion feature, obtains the first confidence score and outputs it.

[0039] As an optional implementation of the embodiment of the present disclosure, the first acoustic feature includes at least one of the following: a tone feature, a drag feature, and a pronunciation duration; and the first text feature includes a semantic integrity feature.

[0040] As an optional implementation of the embodiment of the present disclosure, the acquisition module acquires the first valid voice segment of the voice frame to be recognized, specifically for:

[0041] Obtain the speech frame to be recognized;

[0042] Extracting audio feature information of the speech frame to be recognized;

[0043] Audio clustering is performed according to the audio feature information to obtain the first valid voice segment.

[0044] As an optional implementation of the embodiment of the present disclosure, the acquisition module identifies the first text content corresponding to the first valid voice segment, specifically for:

[0045] Preprocessing the first valid speech segment, wherein the preprocessing includes at least one of the following: denoising, voice extraction, and voice enhancement;

[0046] The preprocessed first valid voice segment is subjected to voice recognition and converted into corresponding first text content.

[0047] As an optional implementation of the embodiment of the present disclosure, the adjustment module is specifically used to:

[0048] When the first confidence score is greater than or equal to a first preset threshold, adjusting the preset silence duration to a first silence duration;

[0049] When the first confidence score is greater than or equal to a second preset threshold and less than the first preset threshold, adjusting the preset silence duration to a second silence duration;

[0050] When the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted.

[0051] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the first silence duration, the second valid voice segment is input into the preset voice recognition model, and a second confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0052] According to the second confidence score, adjusting the first remaining silence duration to the target silence duration; the first remaining silence duration is the difference between the first silence duration and the first detection duration, and the first detection duration is the duration between the start time of the first silence duration and the time when the second valid voice segment is detected;

[0053] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0054] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the second silence duration, the second valid voice segment is input into the preset voice recognition model, and a third confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0055] According to the third confidence score, adjusting the second remaining silence duration to the target silence duration; the second remaining silence duration is the difference between the second silence duration and the second detection duration, and the second detection duration is the duration between the start time of the second silence duration and the time when the second valid voice segment is detected;

[0056] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0057] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the preset silence duration, the second valid voice segment is input into the preset voice recognition model, and a fourth confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0058] According to the fourth confidence score, adjusting the third remaining silence duration to the target silence duration; the third remaining silence duration is the difference between the preset silence duration and the third detection duration, and the third detection duration is the duration between the start time of the preset silence duration and the time when the second valid voice segment is detected;

[0059] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0060] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors;

[0061] a storage device for storing one or more programs,

[0062] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the end of a conversation turn as described in any implementation of the first aspect.

[0063] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining the end of a conversation turn as described in any implementation of the first aspect.

[0064] In a fifth aspect, the disclosed embodiment provides a vehicle, comprising: the electronic device as described in the third aspect.

[0065] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art: obtaining a first valid voice segment of a voice frame to be recognized, and identifying a first text content corresponding to the first valid voice segment, and then inputting the first valid voice segment and the first text content into a preset voice recognition model, and obtaining a first confidence score output by the preset voice recognition model, wherein the first confidence score is used to represent the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content, and according to the first confidence score, adjusting the preset silence duration to a target silence duration, wherein the target silence duration is positively correlated with the first confidence score; and determining whether the conversation of the voice frame to be recognized is over based on the target silence duration. By identifying the valid voice segment of the voice frame to be identified and its corresponding first text content to obtain a first confidence score, the voice and text are combined to predict the probability that the user has not finished a sentence, thereby improving the accuracy of the prediction, and then dynamically adjusting the preset silence duration according to the first confidence score. Adjusting the preset silence duration to the target silence duration can take into account the silence waiting time of the user's pause or hesitation, so as to more flexibly judge whether the user's current conversation is over, avoiding the problem in the prior art that the preset silence duration is a fixed value, which will erroneously interrupt the user, and further improving the user's voice interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0068] Figure 1 It is a flow chart of voice interaction in a vehicle cabin in the prior art;

[0069] Figure 2 It is a flowchart of a method for determining the end of a conversation turn provided by an embodiment of the present disclosure;

[0070] Figure 3 A schematic diagram of the structure of a preset speech recognition model provided in an embodiment of the present disclosure;

[0071] Figure 4 It is a flowchart of another method for determining the end of a conversation turn provided by an embodiment of the present disclosure;

[0072] Figure 5 It is a structural diagram of a device for judging the end of a conversation turn provided by an embodiment of the present disclosure;

[0073] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0074] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0075] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0076] Relational terms such as “first” and “second” in the description and claims of the present disclosure are merely used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0077] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "multiple" refers to two or more.

[0078] Car assistant: In the embodiment of the present disclosure, it can be understood as a voice assistant, that is, through technical means such as voice wake-up, voice detection, voice recognition, semantic understanding, etc., users can conveniently use voice input devices such as microphone arrays, and can use natural language to interact with the car assistant, thereby realizing the combination of the car assistant and intelligent applications and the integration of functions, and further realizing the control of the car system.

[0079] Voice activity detection (VAD) is a technology used in speech processing, which aims to detect whether a speech signal exists. VAD technology is used to determine whether the current speech is a human voice, noise or silence. If the current speech is determined to be a valid human voice, the current speech is input into the speech recognition module. If it is determined to be noise, the current speech is collected and not input into the recognition module.

[0080] End of Turn (EOT): Determine whether a sentence has been finished during the interaction. For example, during a face-to-face communication between two people, one party can judge whether the other party has finished speaking by the content of the other party's speech, the tone of voice at the end, the look in the eyes, or even some slight body movements and expressions, that is, whether he should take over the conversation turn. In the process of human-computer interaction, the computer also needs to judge whether the other party has finished speaking based on the signal in the communication carrier.

[0081] The interaction within the vehicle cabin is mainly based on voice interaction, which can be achieved through the car assistant. After the user says a sentence, whether the car assistant can respond quickly is an important reference for users to judge whether the interaction is friendly.

[0082] The response speed of the car assistant is an important indicator to measure the quality of the car system. After the user finishes speaking, whether the car assistant can quickly reply and execute is also an important reference for users to judge whether the interaction is friendly. Therefore, in order to reduce the user's waiting time, the VAD judgment threshold for the user to finish speaking is usually set to a smaller value, usually within 1 second. Although a smaller VAD judgment threshold can optimize the response speed of the car assistant, in the actual voice interaction process, the user may hesitate, pause, or think while talking to the car. For example, when initiating navigation, "Navigate to..." may appear, and the user hesitates when saying the destination; when using the in-car media, "Play..." appears, and the user has not thought about the song to listen to. At this time, if the user's pause time exceeds the VAD judgment threshold, the system will judge that the user has finished speaking and enter the subsequent interaction process, which will give the user a feeling that the car assistant is "interrupting the conversation". This method of setting the judgment threshold to a fixed value is not conducive to the car assistant to correctly understand the user's semantics, so that it cannot correctly execute user instructions, resulting in a poor user experience.

[0083] The method of judging whether the user has finished a sentence based on the threshold of silence at the end of the voice needs to make a trade-off between response speed and allowing the user to hesitate, so it is not a friendly interaction solution. The present disclosure adds the judgment of whether the current round of dialogue has ended on the basis of the method of judging whether the user has finished a sentence based on silence. If the user hesitates or thinks, it will be judged that the user has not finished speaking this round, so the silence waiting time of VAD is appropriately increased, thereby better resolving the conflict between response speed and allowing the user to hesitate, making the human-computer voice interaction in the cockpit more friendly.

[0084] In addition, in an actual conversation environment, it is more susceptible to interference from background sounds and noises. For example, in a vehicle cabin environment, in addition to the voice of the person giving the command, it may also include wind noise, tire noise, etc. during the vehicle's driving, or other passengers' chatting, navigation, multimedia music and other background sounds. Therefore, the user voice quality collected by the car assistant is poor, and the accuracy of determining whether the conversation is over is not high simply based on whether the voice is muted.

[0085] In order to solve the technical problems existing in the related art, the present disclosure provides a method for judging the end of a conversation turn, obtaining a first valid voice segment of a voice frame to be recognized, inputting the first valid voice segment into a preset voice recognition model, obtaining a first confidence score output by the preset voice recognition model, wherein the first confidence score is used to represent the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content, and adjusting the preset silence duration to a target silence duration according to the first confidence score, wherein the target silence duration is positively correlated with the first confidence score; based on the target silence duration, determining whether the conversation of the voice frame to be recognized is over. By dynamically adjusting the preset silence duration through the first confidence score and adjusting the preset silence duration to the target silence duration, the user's pause or hesitation silence waiting time can be taken into account, thereby more flexibly judging whether the user's current conversation is over, avoiding the problem that the preset silence duration in the prior art is a fixed value and the user is interrupted by mistake, and further improving the user's voice interaction experience.

[0086] like Figure 2 As shown, Figure 2 A method schematic diagram of a method for determining the end of a conversation turn provided by an embodiment of the present disclosure, the method comprising the following steps S21-S24:

[0087] S21. Acquire a first valid voice segment of a voice frame to be recognized, and recognize a first text content corresponding to the first valid voice segment.

[0088] The speech frame to be recognized represents a segment of speech spoken by the user and the vehicle assistant during the human-computer interaction process. For example, the duration of the speech frame to be recognized can be 10 milliseconds to 30 milliseconds, or other reasonable values. The duration of the speech frame to be recognized is not specifically limited. The duration of the first valid speech segment is less than the duration of the speech frame to be recognized. For example, the duration of the first valid speech segment can be 5 milliseconds to 15 milliseconds, or other reasonable values. The duration of the first valid speech segment is not specifically limited.

[0089] Specifically, in the general VAD technology, the judgment result of valid speech and invalid speech is output on each speech frame to be recognized. The combination of multiple consecutive frames of valid speech will be judged as the user starting to speak. After that, multiple consecutive frames of silence will be judged as the user has finished speaking. VAD detection is performed on the speech frame to be recognized to obtain the first valid speech segment.

[0090] Exemplarily, the user says to the car assistant “Xiao X, please help me navigate…, navigate to XXX address”, where the first valid voice segment is “Xiao X, please help me navigate”.

[0091] In some embodiments, in the process of obtaining the first valid speech segment of the speech frame to be recognized, the speech frame to be recognized is firstly obtained, then the audio feature information of the speech frame to be recognized is extracted, and then audio clustering is performed according to the audio feature information to obtain the first valid audio segment.

[0092] The above-mentioned audio feature information may be short-time spectrum features such as Mel Frequency Cepstral Coefficient (MFCC), Perceptual Linear Prediction (PLP), Filter Banks (FBank), etc., or features extracted based on Time Delay Neural Networks (TDNN) such as identity-vector (i-vector), or voiceprint features, timbre features, pitch features, and sound intensity features. The present disclosure does not impose any restrictions on this.

[0093] The clustering algorithm may include: a distance-based clustering algorithm or a density-based clustering algorithm, etc., which is not specifically limited in the present disclosure. For example, Spectral Clustering clustering algorithm, K-Means clustering, mean shift clustering, Gaussian Mixed Model (GMM) Expectation-Maximization (EM) clustering, agglomerative hierarchical clustering, graph community detection, etc. may be used.

[0094] Exemplarily, assuming that the audio feature information is a voiceprint feature, voices with the same voiceprint feature are clustered together to obtain the first valid audio segment of the same person.

[0095] In some embodiments, in the process of identifying the first text content corresponding to the first valid voice segment, the first valid voice segment is preprocessed, and the preprocessing includes but is not limited to at least one of the following: denoising, voice extraction, voice enhancement, etc. Further, the preprocessed first valid voice segment is recognized by automatic speech recognition (Automatic Speech Recognition, ASR) technology to obtain the corresponding first text content, realize speech-to-text conversion, and obtain the text information of the first valid voice segment.

[0096] S22: Input the first valid voice segment and the first text content into a preset voice recognition model, and obtain a first confidence score output by the preset voice recognition model.

[0097] The first confidence score is used to represent the probability that the user has not finished a sentence, predicted based on the first valid voice segment and the first text content.

[0098] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a preset speech recognition model provided in an embodiment of the present disclosure, which shows that the preset speech recognition model includes an acoustic feature encoder 301, a text feature encoder 302, a feature fuser 303 and a classifier 304.

[0099] In some embodiments, the preset speech recognition model extracts acoustic features in the valid speech segment according to the input valid speech segment, and the acoustic features include but are not limited to: tone features, dragging features, and pronunciation duration. It is understandable that the acoustic features of the voice tail of the valid speech segment, such as tone changes, whether there is dragging, pronunciation duration and other information, can also capture the semantic information of the valid speech segment to comprehensively judge whether the user has finished a sentence; and the preset speech recognition model extracts the text features of the text content according to the input text content, optionally, the text content is preprocessed first to obtain the word sequence of the text content, the word sequence is mapped to a word vector and annotated, and then after the cyclic sliding window and undersampling processing as the input of the neural network, the text features of the text content are output by the double-layer Bi-LSTM cyclic network, and the text features include semantic integrity features. Wherein, the neural network can be a double-layer bidirectional long short time (Bi-directional Long Short-Term Memory, Bi-LSTM) cyclic network, and the present disclosure is not specifically limited to this.

[0100] The above step S21 obtains the first valid voice segment of the voice frame to be recognized and the first text content, wherein the first valid voice segment includes the acoustic features of the voice frame to be recognized, and the first text content includes the text features of the voice frame to be recognized, the acoustic features can characterize the integrity of the conversation from the perspective of audio acoustics, and the text features characterize the integrity of the conversation from the perspective of text. It can be understood that the preset voice recognition module determines whether the user has finished a sentence by the acoustic features corresponding to the tone of the voice frame to be recognized and the text features corresponding to the semantics.

[0101] Furthermore, the acoustic features and text features are fused to obtain fused features, and then the fused features are classified by confidence to obtain a confidence score. The confidence score indicates the probability that the user has not finished a sentence, and the value range of the confidence score is 0 to 1. In the process of confidence classification, the fused features can be verified in different dimensions and granularities, such as phoneme, syllable, and full word verification.

[0102] Specifically, the valid speech segment is input into the acoustic feature encoder to obtain the acoustic features output by the acoustic feature encoder, and the text content is input into the text feature encoder to obtain the text features output by the text feature encoder, and then the acoustic features and text features are input into the feature fusion device to obtain the fusion features output by the feature fusion device, and then the fusion features are input into the classifier, and the classifier performs confidence classification according to the fusion features to output the confidence reading score. In this way, the acoustic features and text features of the speech segment are combined to more accurately analyze whether the user has finished a sentence.

[0103] Exemplarily, “Xiao X, please help me navigate” is input into a preset speech recognition model, and it is assumed that the first confidence score output by the preset speech recognition model is 0.9.

[0104] S23: Adjust the preset silence duration to the target silence duration according to the first confidence score.

[0105] The preset silence duration is, that is, the silence threshold when the VAD determines whether the user has finished speaking a sentence. For example, in general, the preset silence duration can be set to 1 second.

[0106] Specifically, since the confidence score indicates the probability that the user has not finished a sentence, it can be understood that the higher the confidence score, the greater the possibility that the user has not finished the sentence; the lower the confidence score, the smaller the possibility that the user has not finished the sentence.

[0107] In some embodiments, reference Figure 4 As shown, Figure 4A flowchart of another method for determining the end of a conversation turn provided in an embodiment of the present disclosure is provided. Step S23 (adjusting the preset silence duration to the target silence duration according to the first confidence score) can be implemented as follows:

[0108] S231: When the first confidence score is greater than or equal to a first preset threshold, adjust the preset silence duration to a first silence duration.

[0109] S232: When the first confidence score is greater than or equal to a second preset threshold and less than the first preset threshold, adjust the preset silence duration to a second silence duration.

[0110] S233: When the first confidence score is less than the second preset threshold, do not adjust the preset silence duration.

[0111] Specifically, since the first preset threshold is greater than the second preset threshold, and the higher the confidence score, the greater the possibility that the user has not finished the sentence; the lower the confidence score, the smaller the possibility that the user has not finished the sentence; therefore, the first silence duration is set to be greater than the second silence duration. When the first confidence score is greater than the first preset threshold, the preset silence duration is adjusted to the first silence duration; when the first confidence score is greater than or equal to the second preset threshold, and less than the first preset threshold, the preset silence duration is adjusted to the second silence duration; when the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted.

[0112] Exemplarily, assume that the preset silence duration is 1 second, the first preset threshold is 0.8, and the second preset threshold is 0.6. When the first confidence score is 0.88, the first confidence score is greater than the first preset threshold. At this time, the preset silence duration is adjusted to the first preset duration of 3 seconds. When the first confidence score is 0.75, the first confidence score is greater than the second preset threshold, and less than the first preset threshold. The preset silence duration is adjusted to the second preset duration of 2 seconds. When the first confidence score is 0.3, the first confidence score is less than the second preset threshold. At this time, the preset silence duration is not adjusted and remains at 1 second.

[0113] S24: Based on the target silence duration, determine whether the conversation of the voice frame to be recognized is ended.

[0114] Specifically, by judging whether a valid voice segment is detected again within the target silence duration, for example, if a valid voice segment is detected again within the target silence duration, it is determined that the conversation of the voice frame to be recognized has not ended; if no valid voice segment is detected within the target silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0115] For example, the user says to the car assistant "Xiao X, please help me navigate..., navigate to XXX address", where the first valid voice segment is "Xiao X, please help me navigate". Assuming the target silence duration is 2 seconds, since the user may hesitate or hesitate for a few seconds after the first valid voice segment, if the valid voice segment "Navigate to XXX address" is detected within 2 seconds, it is determined that the conversation of the voice frame to be recognized has not ended; if the valid voice segment "Navigate to XXX address" is not detected within 2 seconds, it is determined that the conversation of the voice frame to be recognized has ended.

[0116] The method for judging the end of a conversation turn provided by the present disclosure obtains a first valid voice segment of a voice frame to be recognized, inputs the first valid voice segment into a preset voice recognition model, obtains a first confidence score output by the preset voice recognition model, wherein the first confidence score is used to indicate the probability that the user has not finished a sentence, and adjusts the preset silence duration to a target silence duration according to the first confidence score, wherein the target silence duration is positively correlated with the first confidence score; based on the target silence duration, determines whether the conversation of the voice frame to be recognized has ended. By dynamically adjusting the preset silence duration through the first confidence score and adjusting the preset silence duration to the target silence duration, the silent waiting time of the user's pause or hesitation can be taken into account, thereby more flexibly judging whether the user's current conversation has ended, avoiding the problem in the prior art that the preset silence duration is a fixed value and the user is interrupted by mistake, and further improving the user's voice interaction experience.

[0117] In some embodiments, when the target silence duration is a first silence duration, and the first silence duration is greater than the preset silence duration, step S24 (determining whether the conversation of the to-be-recognized voice frame is ended based on the target silence duration) can be implemented as follows:

[0118] If a second valid voice segment is detected within the first silence duration, determining that the conversation of the voice frame to be recognized has not ended;

[0119] If the second valid voice segment is not detected within the first silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0120] The second valid voice segment refers to a valid voice segment detected after the first valid voice segment.

[0121] For example, the user says to the car assistant "Xiao X, please help me navigate..., navigate to XXX address", where the first valid voice segment is "Xiao X, please help me navigate". The second valid voice segment is "Navigate to XXX address".

[0122] Specifically, since the target silence duration is adjusted according to the first confidence score, when the first confidence score is different, the target silence duration is also different. When the first confidence score is greater than the first preset threshold, the preset silence duration is adjusted to the first silence duration, and when the target silence duration is the first silence duration, the first silence duration is greater than the preset silence duration. If a second valid voice segment is detected within the first silence duration, it is determined that the conversation of the voice frame to be recognized has not ended; if the second valid voice segment is not detected within the first silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0123] Exemplarily, assume that the preset silence duration is 1 second, the first preset threshold is 0.8, and the second preset threshold is 0.6. When the first confidence score is 0.9, assume that the preset silence duration is adjusted to the first silence duration of 3 seconds. If a valid voice segment "Navigate to XXX address" is detected within 3 seconds, it is determined that the conversation of the voice frame to be recognized has not ended; if no valid voice segment "Navigate to XXX address" is detected within 3 seconds, it is determined that the conversation of the voice frame to be recognized has ended, and at this time, the car assistant exits the voice interaction.

[0124] Furthermore, if a second valid voice segment is detected within the first silence duration, the second valid voice segment is input into the preset voice recognition model, and a second confidence score of the second valid voice segment output by the preset voice recognition model is obtained; according to the second confidence score, the first remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the voice frame to be recognized is ended.

[0125] Among them, the first remaining silence duration is the difference between the first silence duration and the first detection duration, and the first detection duration is the duration from the start time of the first silence duration to the time when the second valid voice segment is detected.

[0126] Specifically, if valid speech is detected again within the first silence duration, that is, the second valid speech segment, the second valid speech segment will continue to be input into the preset speech recognition model. At this time, the preset speech model will output the second confidence score of the second valid speech segment; according to the second confidence score, the preset silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. That is, according to the second confidence score, the first remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. The specific implementation method is the same as steps S23-S24. When the voice assistant determines that the user's conversation has not ended, it will not interrupt the user. The judgment is repeated in sequence until it is determined that the current round of conversation is over.

[0127] Exemplarily, assume that the preset silence duration is 1 second, the first silence duration is 3 seconds, the first preset threshold is 0.8, and the second preset threshold is 0.6. If within 3 seconds, for example, at the 2.5th second, the second valid voice segment "Navigate to XX City..." is detected, assuming that the second confidence score is 0.9 at this time, then at the moment when the second confidence score is obtained (for example, it can be the 2.6th second, it should be noted that the moment when the second valid voice segment is detected can be the same as the moment when the second confidence score is obtained, or it can be 0.1 seconds different, or set to other reasonable values, which are not specifically limited here.), the first remaining silence duration is adjusted from 0.4 seconds to 3 seconds, and it is continued to be determined within 3 seconds whether this round of conversation is over. For example, at the 2.5th second, the second valid voice segment "Navigate to XX District, XX City..." is detected. Assuming that the second confidence score is 0.75 at this time, when the second confidence score is obtained (for example, it can be the 2.6th second), the first remaining silence duration is adjusted from 0.4 seconds to 2 seconds, and the judgment on whether the current round of conversation is over continues within 2 seconds. For example, at the 2.5th second, the second valid voice segment "Navigate to XX Company, XX District, XX City" is detected. Assuming that the second confidence score is 0.3 at this time, when the second confidence score is obtained (for example, it can be the 2.6th second), the first remaining silence duration is adjusted from 0.4 seconds to 1 second, and the judgment on whether the current round of conversation is over continues within 1 second.

[0128] In some embodiments, when the target silence duration is a second silence duration, the second silence duration is greater than the preset silence duration and less than the first silence duration, step S24 (determining whether the conversation of the voice frame to be recognized is ended based on the target silence duration) can be implemented in the following manner:

[0129] If a second valid voice segment is detected within the second silence duration, determining that the conversation of the voice frame to be recognized has not ended;

[0130] If the second valid voice segment is not detected within the second silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0131] Specifically, since the target silence duration is adjusted according to the first confidence score, when the first confidence score is different, the target silence duration is also different. When the first confidence is greater than or equal to the second preset threshold value, and less than the first preset threshold value, the preset silence duration is adjusted to the second silence duration. When the target silence duration is the second silence duration, the second silence duration is greater than the preset silence duration, and less than the first silence duration. If a second valid voice segment is detected within the second silence duration, it is determined that the conversation of the voice frame to be recognized has not ended; if the second valid voice segment is not detected within the second silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0132] Exemplarily, assume that the preset silence duration is 1 second, the first preset threshold is 0.8, and the second preset threshold is 0.6. When the first confidence score is 0.7, assume that the preset silence duration is adjusted to the second silence duration of 2 seconds. If a valid voice segment "Navigate to XXX address" is detected within 2 seconds, it is determined that the conversation of the voice frame to be recognized has not ended; if no valid voice segment "Navigate to XXX address" is detected within 2 seconds, it is determined that the conversation of the voice frame to be recognized has ended, and at this time, the car assistant exits the voice interaction.

[0133] Furthermore, if a second valid voice segment is detected within the second silence duration, the second valid voice segment is input into the preset voice recognition model to obtain a third confidence score of the second valid voice segment output by the preset voice recognition model; the second remaining silence duration is adjusted to the target silence duration; and based on the target silence duration, it is determined whether the conversation of the voice frame to be recognized is ended.

[0134] Among them, the second remaining silence duration is the difference between the second silence duration and the second detection duration, and the second detection duration is the duration from the start time of the second silence duration to the time when the second valid voice segment is detected.

[0135] Specifically, if valid speech is detected again within the second silence duration, that is, the second valid speech segment, the second valid speech segment will continue to be input into the preset speech recognition model. At this time, the preset speech model will output the third confidence score of the second valid speech segment. According to the third confidence score, the second remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. That is, according to the third confidence score, the second remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. The specific implementation method is the same as steps S23-S24. When the voice assistant determines that the user's conversation has not ended, it will not interrupt the user. The judgment is repeated in sequence until it is determined that the current round of conversation is over.

[0136] Exemplarily, assume that the preset silence duration is 1 second, the second silence duration is 2 seconds, the first preset threshold is 0.8, and the second preset threshold is 0.6. If within 2 seconds, for example, at the 1.3th second, the second valid voice segment "Navigate to XX City..." is detected, assuming that the third confidence score is 0.9 at this time, then at the moment when the third confidence score is obtained (for example, it can be the 1.4th second, it should be noted that the moment when the second valid voice segment is detected can be the same as the moment when the third confidence score is obtained, or it can be 0.1 seconds apart, or set to other reasonable values, which are not specifically limited here.), the second remaining silence duration is adjusted from 0.6 seconds to 3 seconds, and it is continued to be determined within 3 seconds whether this round of conversation is over. For example, at the 1.3th second, the second valid voice segment "Navigate to XX District, XX City..." is detected. Assuming that the third confidence score is 0.75 at this time, at the moment when the third confidence score is obtained (for example, it can be the 1.4th second), the second remaining silence duration is adjusted from 0.6 seconds to 2 seconds, and the judgment on whether the current round of conversation is over continues within 2 seconds. For example, at the 1.3th second, the second valid voice segment "Navigate to XX Company, XX District, XX City" is detected. Assuming that the third confidence score is 0.3 at this time, at the moment when the third confidence score is obtained (for example, it can be the 1.4th second), the second remaining silence duration is adjusted from 0.6 seconds to 1 second, and the judgment on whether the current round of conversation is over continues within 1 second.

[0137] In some embodiments, when the target silence duration is the preset silence duration, step S24 (determining whether the conversation of the to-be-recognized voice frame is ended based on the target silence duration) can be implemented as follows:

[0138] If a second valid voice segment is detected within the preset silence duration, determining that the conversation of the voice frame to be recognized has not ended;

[0139] If the second valid voice segment is not detected within the preset silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0140] Specifically, since the target silence duration is adjusted according to the first confidence score, when the first confidence score is different, the target silence duration is also different. When the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted. At this time, the target silence duration is the preset silence duration. If a second valid voice segment is detected within the preset silence duration, it is determined that the conversation of the voice frame to be recognized has not ended; if the second valid voice segment is not detected within the preset silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0141] Exemplarily, assume that the preset silence duration is 1 second, the first preset threshold is 0.8, and the second preset threshold is 0.6. When the first confidence score is 0.5, since the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted. If a valid voice segment "Navigate to XXX address" is detected within 1 second, it is determined that the conversation of the voice frame to be recognized has not ended; if no valid voice segment "Navigate to XXX address" is detected within 1 second, it is determined that the conversation of the voice frame to be recognized has ended, and at this time, the car assistant exits the voice interaction.

[0142] Furthermore, if a second valid voice segment is detected within the preset silence duration, the second valid voice segment is input into the preset voice recognition model, and a fourth confidence score of the second valid voice segment output by the preset voice recognition model is obtained; according to the fourth confidence score, the third remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the voice frame to be recognized is ended.

[0143] Among them, the third remaining silence duration is the difference between the preset silence duration and the third detection duration, and the third detection duration is the duration between the starting moment of the preset silence duration and the moment when the second valid voice segment is detected.

[0144] Specifically, if valid speech is detected again within the preset silence duration, that is, the second valid speech segment, the second valid speech segment will continue to be input into the preset speech recognition model. At this time, the preset speech model will output the fourth confidence score of the second valid speech segment; according to the fourth confidence score, the third remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. That is, according to the fourth confidence score, the third remaining silence duration is adjusted to the target silence duration; based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is over. The specific implementation method is the same as steps S23-S24. When the voice assistant determines that the user's conversation has not ended, it will not interrupt the user. The judgment is repeated in sequence until it is determined that the current round of conversation is over.

[0145] Exemplarily, it is assumed that the preset silence duration is 1 second, the first preset threshold is 0.8, and the second preset threshold is 0.6. If within 1 second, for example, at the 0.6th second, the second valid voice segment "Navigate to XX City..." is detected, assuming that the fourth confidence score is 0.9 at this time, then at the moment when the fourth confidence score is obtained (for example, it can be the 0.7th second, it should be noted that the moment when the second valid voice segment is detected can be the same as the moment when the fourth confidence score is obtained, or it can be 0.1 seconds different, or set to other reasonable values, which are not specifically limited here.), the third remaining silence duration is adjusted from 0.3 seconds to 3 seconds, and it is continued to be judged within 3 seconds whether this round of dialogue is over. For example, at the 0.6th second, the second valid voice segment "Navigate to XX District, XX City..." is detected, assuming that the fourth confidence score is 0.75 at this time, then at the moment when the fourth confidence score is obtained (for example, it can be the 0.7th second), the third remaining silence duration is adjusted from 0.3 seconds to 2 seconds, and it is continued to be judged within 2 seconds whether this round of dialogue is over. For example, at the 0.6th second, the second valid voice segment "Navigate to XX Company, XX District, XX City" is detected. Assuming that the fourth confidence score is 0.3 at this time, at the moment when the fourth confidence score is obtained (for example, it may be the 0.7th second), the third remaining silence time is adjusted from 0.3 seconds to 1 second, and it is continued to be determined within 1 second whether this round of conversation is over.

[0146] The method for judging the end of a conversation turn provided by the present disclosure obtains a first valid voice segment of a voice frame to be recognized, inputs the first valid voice segment into a preset voice recognition model, obtains a first confidence score output by the preset voice recognition model, wherein the first confidence score is used to indicate the probability that the user has not finished a sentence, and adjusts the preset silence duration to a target silence duration according to the first confidence score, wherein the target silence duration is positively correlated with the first confidence score; based on the target silence duration, determines whether the conversation of the voice frame to be recognized has ended. By dynamically adjusting the preset silence duration through the first confidence score and adjusting the preset silence duration to the target silence duration, the silent waiting time of the user's pause or hesitation can be taken into account, thereby more flexibly judging whether the user's current conversation has ended, avoiding the problem in the prior art that the preset silence duration is a fixed value and the user is interrupted by mistake, and further improving the user's voice interaction experience.

[0147] In some embodiments, reference Figure 5 As shown, a device for judging the end of a dialogue turn is provided, comprising:

[0148] An acquisition module 510 is used to acquire a first valid speech segment of a speech frame to be recognized;

[0149] An input module 520 is used to input the first valid voice segment into a preset voice recognition model to obtain a first confidence score output by the preset voice recognition model; the first confidence score is used to represent the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content;

[0150] An adjustment module 530, configured to adjust a preset silence duration to a target silence duration according to the first confidence score; the target silence duration is positively correlated with the first confidence score;

[0151] The judgment module 540 is used to determine whether the conversation of the speech frame to be recognized is ended based on the target silence duration.

[0152] As an optional implementation of the embodiment of the present disclosure, the adjustment module is specifically used to:

[0153] When the first confidence score is greater than or equal to a first preset threshold, adjusting the preset silence duration to a first silence duration;

[0154] When the first confidence score is greater than or equal to a second preset threshold and less than the first preset threshold, adjusting the preset silence duration to a second silence duration;

[0155] When the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted.

[0156] As an optional implementation of the embodiment of the present disclosure, when the target silence duration is a first silence duration, and the first silence duration is greater than the preset silence duration, the judgment module is specifically configured to:

[0157] If a second valid voice segment is detected within the first silence duration, it is determined that the conversation of the voice frame to be recognized has not ended; the second valid voice segment represents a valid voice segment detected after the first valid voice segment;

[0158] If the second valid voice segment is not detected within the first silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0159] As an optional implementation of the embodiment of the present disclosure, when the target silence duration is a second silence duration, the second silence duration is greater than the preset silence duration and less than the first silence duration, the judgment module is specifically used to:

[0160] If a second valid voice segment is detected within the second silence duration, determining that the conversation of the voice frame to be recognized has not ended;

[0161] If the second valid voice segment is not detected within the second silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0162] As an optional implementation of the embodiment of the present disclosure, when the target silence duration is the preset silence duration, the judgment module is specifically configured to:

[0163] If a second valid voice segment is detected within the preset silence duration, determining that the conversation of the voice frame to be recognized has not ended;

[0164] If the second valid voice segment is not detected within the preset silence duration, it is determined that the conversation of the voice frame to be recognized has ended.

[0165] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the first silence duration, the second valid voice segment is input into the preset voice recognition model, and a second confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0166] According to the second confidence score, adjusting the first remaining silence duration to the target silence duration; the first remaining silence duration is the difference between the first silence duration and the first detection duration, and the first detection duration is the duration between the start time of the first silence duration and the time when the second valid voice segment is detected;

[0167] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0168] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the second silence duration, the second valid voice segment is input into the preset voice recognition model, and a third confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0169] According to the third confidence score, adjusting the second remaining silence duration to the target silence duration; the second remaining silence duration is the difference between the second silence duration and the second detection duration, and the second detection duration is the duration between the start time of the second silence duration and the time when the second valid voice segment is detected;

[0170] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0171] As an optional implementation of the embodiment of the present disclosure, if a second valid voice segment is detected within the preset silence duration, the second valid voice segment is input into the preset voice recognition model, and a fourth confidence score of the second valid voice segment output by the preset voice recognition model is obtained;

[0172] According to the fourth confidence score, adjusting the third remaining silence duration to the target silence duration; the third remaining silence duration is the difference between the preset silence duration and the third detection duration, and the third detection duration is the duration between the start time of the preset silence duration and the time when the second valid voice segment is detected;

[0173] Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

[0174] The present disclosure provides a device for judging the end of a conversation turn, which obtains a first valid voice segment of a voice frame to be recognized, inputs the first valid voice segment into a preset voice recognition model, obtains a first confidence score output by the preset voice recognition model, wherein the first confidence score is used to indicate the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content, and adjusts the preset silence duration to a target silence duration based on the first confidence score, wherein the target silence duration is positively correlated with the first confidence score; based on the target silence duration, determines whether the conversation of the voice frame to be recognized has ended. By dynamically adjusting the preset silence duration through the first confidence score and adjusting the preset silence duration to the target silence duration, the user's pause or hesitation silence waiting time can be taken into account, thereby more flexibly judging whether the user's current conversation has ended, avoiding the problem that the preset silence duration in the prior art is a fixed value and the user is interrupted by mistake, and further improving the user's voice interaction experience.

[0175] For the specific definition of the conversation turn end judgment device, please refer to the definition of the conversation turn end judgment method above, which will not be repeated here. Each module in the above conversation turn end judgment device can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor of the electronic device in the form of hardware, or can be stored in the processor of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0176] The present disclosure also provides an electronic device, Figure 5 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 5As shown, the electronic device provided in this embodiment includes: a memory 51 and a processor 52, the memory 51 is used to store a computer program; the processor 52 is used to execute the steps executed in any embodiment of the fault identification method of the image acquisition device provided by the above method embodiment when calling the computer program. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a method for fault identification of an image acquisition device is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0177] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] In some embodiments, the conversation turn end judgment device provided by the present disclosure can be implemented in the form of a computer program. Figure 5 The memory of the electronic device may store various program modules constituting the conversation turn end judgment device of the electronic device, for example, Figure 4 The acquisition module 410, input module 420, adjustment module 430, and judgment module 440 shown in the figure. The computer program composed of various program modules enables the processor to execute the steps of the fault identification method of the image acquisition device of the electronic device of each embodiment of the present disclosure described in this specification.

[0179] The embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the fault identification method for the image acquisition device provided by the above method embodiment is implemented.

[0180] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0181] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0182] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0183] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0184] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0185] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining the end of a conversation turn, characterized in that: include: Acquire a first valid voice segment of a voice frame to be recognized, and recognize a first text content corresponding to the first valid voice segment; Inputting the first valid voice segment and the first text content into a preset voice recognition model, and obtaining a first confidence score output by the preset voice recognition model; the first confidence score is used to indicate the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content; According to the first confidence score, adjusting the preset silence duration to the target silence duration; The target silence duration is positively correlated with the first confidence score; Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

2. The method according to claim 1, characterized in that The process of presetting the first confidence score output by the speech recognition model includes: The preset speech recognition model extracts a first acoustic feature of the first valid speech segment, and extracts a first text feature of the first text content; The preset speech recognition model performs feature fusion on the first acoustic feature and the first text feature to obtain a first fusion feature; The preset speech recognition model performs confidence classification on the first fusion feature, obtains the first confidence score and outputs it.

3. The method according to claim 2, characterized in that The first acoustic feature includes at least one of the following: a tone feature, a drag feature, and a pronunciation duration; the first text feature includes a semantic integrity feature.

4. The method according to claim 1, characterized in that: Obtaining the first valid speech segment of the speech frame to be recognized, including: Obtain the speech frame to be recognized; Extracting audio feature information of the speech frame to be recognized; Audio clustering is performed according to the audio feature information to obtain the first valid voice segment.

5. The method according to claim 1, characterized in that Identifying first text content corresponding to the first valid voice segment includes: Preprocessing the first valid speech segment, wherein the preprocessing includes at least one of the following: denoising, voice extraction, and voice enhancement; The preprocessed first valid voice segment is subjected to voice recognition and converted into corresponding first text content.

6. The method according to claim 1, characterized in that The step of adjusting the preset silence duration to the target silence duration according to the first confidence score includes: When the first confidence score is greater than or equal to a first preset threshold, adjusting the preset silence duration to a first silence duration; When the first confidence score is greater than or equal to a second preset threshold and less than the first preset threshold, adjusting the preset silence duration to a second silence duration; When the first confidence score is less than the second preset threshold, the preset silence duration is not adjusted.

7. The method according to claim 6, characterized in that The method further comprises: If a second valid voice segment is detected within the first silence duration, inputting the second valid voice segment into the preset voice recognition model, and obtaining a second confidence score of the second valid voice segment output by the preset voice recognition model; According to the second confidence score, adjusting the first remaining silence duration to the target silence duration; the first remaining silence duration is the difference between the first silence duration and the first detection duration, and the first detection duration is the duration between the start time of the first silence duration and the time when the second valid voice segment is detected; Based on the target silence duration, it is determined whether the conversation of the speech frame to be recognized is ended.

8. A device for determining the end of a conversation turn, characterized in that: include: An acquisition module, used to acquire a first valid voice segment of a voice frame to be recognized, and recognize a first text content corresponding to the first valid voice segment; An input module, used to input the first valid voice segment and the first text content into a preset voice recognition model, and obtain a first confidence score output by the preset voice recognition model; the first confidence score is used to represent the probability that the user has not finished a sentence predicted based on the first valid voice segment and the first text content; An adjustment module, configured to adjust the preset silence duration to a target silence duration according to the first confidence score; The target silence duration is positively correlated with the first confidence score; The judgment module is used to determine whether the conversation of the speech frame to be recognized is ended based on the target silence duration.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the end of a dialogue turn as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining the end of a dialogue turn as claimed in any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: The device for determining the end of a conversation turn as claimed in claim 8, or the electronic device as claimed in claim 9.

Images