Question-answer processing methods and apparatus, device, storage medium and program product

Abstract

Question-answer processing methods and apparatus, a device, a storage medium and a program product. A method comprises: at a client, in response to receiving a question speech from a user, sending to a server the question speech (410); determining a dialog scenario corresponding to the question speech (420); sending to the server an indication of the dialog scenario or a response strategy of the dialog scenario (430); and receiving from the server a question text and / or a response to the question speech (440), the response strategy comprising at least one of the following: a speech processing strategy for the question speech and a response processing strategy for the question speech, the question text being determined on the basis of the speech processing strategy and the question speech, and / or the response being determined on the basis of the question text and the response processing strategy.

Description

Method, device, equipment, storage medium and program product for question and answer processing

[0001] The present application claims priority to the Chinese patent application No. 202410870225.6, filed on June 28, 2024, and entitled "Method, device, equipment, storage medium and program product for question and answer processing", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The example embodiments of the present disclosure generally relate to the field of computer, and in particular, to a method, device, electronic equipment, computer readable storage medium and computer program product for question and answer processing. BACKGROUND

[0003] With the development of machine learning technology, various types of product forms have emerged, one of which is a voice question and answer product that supports voice dialogue. However, the dialogue of the voice question and answer product in a noisy environment is easy to be interrupted, and the dialogue quality still needs to be improved. SUMMARY

[0004] In a first aspect of the present disclosure, a method for question and answer processing is provided. The method comprises: at a client, in response to receiving a question voice of a user, sending the question voice to a server; determining a dialogue scenario corresponding to the question voice; sending an indication of the dialogue scenario or a response strategy of the dialogue scenario to the server, the response strategy including at least one of a voice processing strategy for the question voice and a response processing strategy for the question voice; and receiving a question text and / or a response for the question voice from the server, wherein the question text is determined based on the voice processing strategy and the question voice, and / or the response is determined based on the question text and the response processing strategy.

[0005] In a second aspect of the present disclosure, a method for question and answer processing is provided. The method comprises: at a server, receiving a question voice of a user from a client; receiving an indication of a dialogue scenario corresponding to the question voice or a response strategy of the dialogue scenario from the client, the response strategy including at least one of a voice processing strategy for the question voice and a response processing strategy for the question voice; based on the response strategy, performing at least one of the following: obtaining a question text generated based on the voice processing strategy and the question voice, or determining a response for the question voice based on the question text and the response processing strategy; and sending the question text and / or the response to the client.

[0006] In a third aspect of the present disclosure, an apparatus for question and answer processing is provided. The apparatus includes: a question voice sending module configured to send, at a client, a question voice of a user to a server in response to receiving the question voice; a dialogue scenario determining module configured to determine a dialogue scenario corresponding to the question voice; an indication strategy sending module configured to send, to the server, an indication of the dialogue scenario or a response strategy of the dialogue scenario, the response strategy including at least one of: a voice processing strategy for the question voice, a response processing strategy for the question voice; and a response receiving module configured to receive, from the server, a question text and / or a response for the question voice, wherein the question text is determined based on the voice processing strategy and the question voice, and / or the response is determined based on the question text and the response processing strategy.

[0007] In a fourth aspect of the present disclosure, an apparatus for question and answer processing is provided. The apparatus includes: a question voice receiving module configured to receive, at a server, a question voice of a user from a client; an indication strategy receiving module configured to receive, from the client, an indication of a dialogue scenario corresponding to the question voice or a response strategy of the dialogue scenario, the response strategy including at least one of: a voice processing strategy for the question voice, a response processing strategy for the question voice; a response strategy executing module configured to perform at least one of: obtaining a question text generated based on the voice processing strategy and the question voice, or determining a response for the question voice based on the question text and the response processing strategy, based on the response strategy; and a response sending module configured to send, to the client, the question text and / or the response.

[0008] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the electronic device to perform the method of the first aspect or the method of the second aspect.

[0009] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided. The medium has stored thereon computer-executable instructions that, when executed by a processor, implement the method of the first aspect or the method of the second aspect.

[0010] In a seventh aspect of the present disclosure, a computer program product is provided. The product includes computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method of the first aspect or the method of the second aspect according to the present disclosure.

[0011] It should be understood that the content described in this section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0014] Figure 2 illustrates a flowchart of a signaling flow for question-and-answer processing according to some embodiments of the present disclosure;

[0015] Figure 3 illustrates a schematic diagram of an example architecture for question-answering processing according to some embodiments of the present disclosure;

[0016] Figure 4 shows a flowchart of a question-and-answer processing procedure according to some embodiments of the present disclosure;

[0017] Figure 5 shows a flowchart of a question-and-answer process according to some embodiments of the present disclosure;

[0018] Figure 6 shows an exemplary structural block diagram of an apparatus for question-and-answer processing according to some embodiments of the present disclosure;

[0019] Figure 7 illustrates an exemplary structural block diagram of an apparatus for question-and-answer processing according to some embodiments of the present disclosure; and

[0020] Figure 8 shows a block diagram of an electronic device that can implement one or more embodiments of the present disclosure. Detailed Implementation

[0021] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0023] In this document, unless explicitly stated otherwise, performing a step "in response to" an action means performing the step after the action, but can include one or more intervening steps.

[0024] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the obtaining, use, storage or deletion of the data) should comply with the requirements of the relevant laws and regulations and relevant provisions.

[0025] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the type of information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the relevant user and the authorization of the relevant user should be obtained through appropriate means according to the relevant laws and regulations, wherein the relevant user can include any type of right subject, such as an individual, an enterprise or a group.

[0026] For example, in response to receiving the active request of the user, a prompt information is sent to the relevant user to explicitly prompt the relevant user that the operation requested to be performed will need to obtain and use the information of the relevant user, so that the relevant user can voluntarily choose whether to provide the information to the software or hardware such as the electronic device, the application program, the server or the storage medium performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0027] As an optional but non-limiting implementation manner, in response to receiving the active request of the relevant user, the prompt information can be sent to the relevant user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide the information to the electronic device.

[0028] It can be understood that the above notification and user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0029] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that the corresponding output can be generated for a given input after the training is completed. The generation of the model can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. The neural network model is an example of a model based on deep learning. In this document, "model" can also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably herein.

[0030] A“neural network” is a machine learning network based on deep learning. A neural network is capable of processing an input and providing a corresponding output, which typically includes an input layer and an output layer and one or more hidden layers between the input layer and the output layer. Neural networks used in deep learning applications typically include many hidden layers, increasing the depth of the network. The layers of a neural network are connected in sequence, such that the output of a preceding layer is provided as input to a following layer, with the input layer receiving the input to the neural network and the output of the output layer as the final output of the neural network. Each layer of the neural network includes one or more nodes (also referred to as processing nodes or neurons), each of which processes input from the previous layer.

[0031] Generally, machine learning can include three stages, namely a training stage, a testing stage, and an application stage (also referred to as an inference stage). In the training stage, a given model can be trained using a large amount of training data, iteratively updating parameter values until the model is able to derive consistent inferences from the training data that meet an expected objective. Through training, the model can be considered to have learned an association (also referred to as a mapping) from input to output from the training data. The parameter values of the trained model are determined. In the testing stage, test inputs are applied to the trained model to test whether the model is able to provide correct outputs, determining the performance of the model. In the application stage, the model can be used to process actual inputs based on the trained parameter values to determine corresponding outputs.

[0032] FIG. 1 illustrates a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In this example environment 100, an application 120 is installed in a client 110. A user 140 can interact with the application 120 via the client 110 and / or an attached device of the client 110. For example, the application 120 can capture a voice 145 of the user 140 via a voice capture device (e.g., a microphone) of the client 110.

[0033] In embodiments of the present disclosure, the application 120 can be any suitable application with human-to-computer dialog functionality. For example, the application 120 can provide a digital assistant for human-to-computer dialog. The digital assistant supports text dialog services, voice dialog services, and content dialog in other modalities with the user 140. In some embodiments, the application 120 or the digital assistant therein can utilize the machine learning model 160 (which can include one or more machine learning models, e.g., can include machine learning model 160-1, machine learning model 160-2, …, machine learning model 160-N, etc., where N is a positive integer. For ease of description, one or more machine learning models are collectively referred to as the machine learning model 160 herein) to support interactions with the user 140. For example, the application 120 or the digital assistant therein can utilize one or more machine learning models 160 to provide question-and-answer services to the user 140.

[0034] In the environment 100, the client 110 can present a user interface 150 of the application 120 if the application 120 is active. The user interface 150 can include various pages that the application 120 is capable of providing, such as a dialog page between the user and the digital assistant (where current and historical dialog, including text and voice dialog content, can be presented), etc. In some embodiments, the client 110 can play voice 152 and present text 154 in the user interface 150. The voice 152 can include, for example, the voice 145 from the user 140 or the voice of a response to the voice 145.

[0035] The machine learning model 160 can be different types of models. In some embodiments, one or more machine learning models 160 can be built based on a language model (LM). The machine learning model used is a content generative model, capable of generating a corresponding output based on a model input. In some embodiments, the language model based machine learning model is capable of text modalities of model input (e.g., natural language and / or machine language) and / or non-text modalities of model input (e.g., images, voice, video, etc.), and is capable of generating a desired output from the model input and a prompt word. The prompt word here is used to guide the machine learning model to generate a model output that can address a user need indicated by the model input. In the application scenario for supporting user dialog, the input of the user 140 can be provided to the machine learning model 160 as at least a part of the model input (other parts can include the prompt word). The user input is considered as a question. Based on the model output, a corresponding response can be generated to be provided to the user 140.

[0036] In some embodiments, the one or more machine learning models 160 can be speech-related models, including an automatic speech recognition (ASR) model and a text-to-speech (TTS) model. The input of an ASR model is speech, and the output is text. The input of a TTS model is text, and the output is corresponding speech.

[0037] In some embodiments, the client 110 communicates with the server 130 to enable provisioning of services to the application 120. As shown in FIG. 1, the server 130 can invoke the machine learning model 160 to support human-to-computer dialog functionality between the application 120 and the user 140 based on the output of the machine learning model 160. The client 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a tablet computer, a laptop computer, a notebook computer, a netbook computer, a smartbook, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including an apparatus and peripherals or any combination thereof. In some embodiments, the client 110 can also support any type of interface for the user (such as “wearable” circuitry, etc.). The server 130 can be various types of computing systems / servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, etc. The server 130 can be implemented, for example, based on a cloud environment.

[0038] It should be understood that the structure and functionality of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0039] As mentioned previously, with the development of artificial intelligence technology, various types of product forms have emerged, including but not limited to AI text question and answer, AI voice question and answer, and AI text-to-image, etc. AI voice question and answer supports users to dialogue with digital assistants (sometimes also referred to as robots) using voice. Specifically, users can ask questions to digital assistants using voice, and digital assistants can feed back answers to users using voice, text, or other ways.

[0040] Such AI voice question answering products are prone to be interrupted in a noisy environment. For example, in an environment with human voice such as a conference room, a user utters a question to a digital assistant using voice. After the digital assistant receives the user voice, it processes the user voice to generate a response for feeding back to the user. If another piece of user voice in the environment is identified during the process of the digital assistant responding to the user voice, even if the other piece of user voice is not intended for the digital assistant, it can trigger the digital assistant to interrupt the current conversation and generate a response based on the other piece of user voice or based on both pieces of voice. This results in such AI voice question answering products being prone to frequent interruption in an environment with more human voice, affecting normal human-computer conversation of the user.

[0041] In view of this, according to an embodiment of the present disclosure, an improved solution for question answering processing is provided. According to the solution of the present disclosure, in response to receiving a question voice of a user, the question voice is sent to a server; a conversation scenario corresponding to the question voice is determined; an indication of the conversation scenario or a response strategy of the conversation scenario is sent to the server, the response strategy including at least one of the following: a voice processing strategy for the question voice, a response processing strategy for the question voice; and a question text and / or a response to the question voice are received from the server, wherein the question text is determined based on the voice processing strategy and the question voice, and / or the response is determined based on the question text and the response processing strategy.

[0042] In this way, in the application scenario of human-computer conversation using voice, the response strategy can be determined based on the conversation scenario, and the question voice can be responded to based on the response strategy, which can avoid the problem of frequent interruption of human-computer conversation in a noisy environment.

[0043] Some example embodiments of the present disclosure will be described below with continuous reference to the drawings.

[0044] FIG. 2 shows a flowchart of a signaling flow 200 for question answering processing according to some embodiments of the present disclosure. The signaling flow 200 involves the client 110 and the server 130. FIG. 3 shows a schematic diagram of an example architecture 300 for question answering processing according to some embodiments of the present disclosure. For ease of discussion, the signaling flow 200 will be described with reference to the environment of FIG. 1 and in conjunction with the example architecture 300 shown in FIG. 3.

[0045] In an embodiment of the present disclosure, as shown in the signaling flow 200, the client 110 sends (202) the question voice to the server 130 in response to receiving the question voice of the user 140. The server 130 receives (204) the question voice of the user 140 from the client 110.

[0046] As shown in FIG. 3, the client 110 can include a service module 310. The service module 310 can correspond to an application that supports voice conversation. The service module 310 includes an application voice module 311 and an application message module 312. The server 130 can include an access layer 340, a voice module 350, and a question-answer module 360. The access layer 340 includes a network interface 341 and a network interface 342. A first network connection can be established between the network interface 341 of the client 110 and the server 130. A second network connection can be established between the network interface 342 of the client 110 and the server 130. Alternatively or additionally, the first network connection and / or the second network connection can be a long connection in compliance with a transmission control protocol (TCP). Of course, the first network connection and the second network connection are not limited to the long connection in compliance with the TCP protocol, and the first network connection and the second network connection can also be network connections in compliance with other communication protocols. In a specific implementation, a selection configuration can be made according to actual needs.

[0047] The application voice module 311 can receive a voice (for example, the voice 145) from a user (for example, the user 140). In some embodiments, the application voice module 311 can receive a voice (that is, “user voice”, also referred to as “question voice” in this document) from the user 140. The application voice module 311 can send the user voice to the network interface 341 of the server 130 via the first network connection, and send the user voice to the voice module 350 of the server 130 through the network interface 341.

[0048] In the embodiments of the present disclosure, as shown in the signaling flow 200, the client 110 can determine (206) a conversation scenario corresponding to the question voice. The conversation scenario here can be understood as a definition made for the process of human-machine conversation interaction based on one or more information dimensions (for example, time, space, turn, state, content, and the like). For example, the process of conversation interaction can be determined as one or more conversation scenarios based on time. Also for example, the process of conversation interaction can be determined as one or more conversation scenarios based on the state of conversation interaction. Also for example, the process of conversation interaction can be determined as one or more conversation scenarios based on time, content, and children's clothing.

[0049] In some embodiments, the client 110 can determine the conversation scenario corresponding to the question voice based on at least one of the following: comparison of the conversation turn of the question voice in the current conversation interaction with a threshold turn; whether the current conversation interaction in which the question voice is located is a first wake-up after interaction or a non-first wake-up after interaction; and a question-answer state corresponding to the conversation turn of the question voice, the question-answer state including a first state of waiting for receiving a user question or a second state of responding to a user question.

[0050] Exemplarily, in the human-computer conversation application scenario, if the user 140 needs to start a human-computer conversation with the digital assistant, the digital assistant of the human-computer conversation needs to be first woken up, and at this time, it can be considered that a conversation interaction is started. For example, the user 120 can trigger a specific button to open the human-computer conversation interface of the digital assistant, or speak a wake-up word (for example, Xiao A, Xiao B, Xiao C, and the like) to wake up the digital assistant of the human-computer conversation.

[0051] If the client 110 receives a question voice after the digital assistant is first woken up, the current conversation interaction in which the question voice is located can be considered as a first-woken-up interaction. During the human-computer conversation, an interval can occur (for example, the user 140 temporarily stops asking questions), and the digital assistant can enter a standby state or a dormant state, and the like. If the user 140 needs to ask a question again, the digital assistant needs to be woken up again. The operation of waking up / activating the digital assistant for human-computer interaction again can be considered as a non-first-woken-up operation (for example, second-woken-up, third-woken-up, and the like). If the client 110 receives a question voice after the digital assistant is woken up for the non-first time, the current conversation interaction in which the question voice is located can be considered as a non-first-woken-up interaction.

[0052] In a conversation interaction, one or more user voices 145 of the user 140 can be accepted. The digital assistant can feed back an answer to at least one user voice 145. For example, the digital assistant can call the machine learning model 160 to feed back an answer to one user voice 145, or to a plurality of continuous user voices 145. From receiving a first user voice 145 in the at least one user voice 145 to completing the answer feedback to the at least one user voice 145, it can be considered as one conversation turn. The client 110 receives a user voice 145 again, and until the feedback to the received user voice 145 again is completed, it can be considered as completing another conversation turn. The client 110 or the server 130 can configure an identifier for the user voice 145 or the ASR text corresponding to the user voice 145, and indicate the conversation turn of the user voice 145 or the ASR text through the identifier. The client 110 can determine the conversation turn of the user voice 145 based on the identifier. Of course, the client 110 can also determine the conversation turn of the previous user voice 145 based on the identifier of the previous user voice 145, and then determine the conversation turn of the current user voice 145. The threshold turn can be selected and configured according to actual needs, for example, three turns, four turns, five turns, and the like. After determining the conversation turn of the user voice 145, the client 110 can compare the conversation turn of the user voice 145 with the threshold turn, and determine whether the conversation turn of the user voice 145 exceeds the threshold turn (for example, determine whether the conversation turn exceeds 5 times).

[0053] In one dialog turn, if there is no user speech 145 requiring a digital assistant response before the client 110 receives the user speech 145 of the user 140, the client 110 is in a state of waiting for receiving speech in the environment. If the client 110 receives the user speech 145 in this state, the dialog turn corresponding to the question and answer state in which the user speech 145 is located can be a first state (for example, a waiting for receiving state, etc.). If the digital assistant is determining a response to a previous user speech 145, or the digital assistant is presenting and / or playing a response to a previous user speech 145, and the client 110 receives the user speech 145 again in this state, the question and answer state of the dialog turn in which the user speech 145 is located can be considered as a second state (for example, a thinking state, a reply state, or a response state, etc.).

[0054] If the user 140 has no more questions, the current dialog interaction can be ended by, for example, closing the human-computer dialogue interface of the digital assistant, triggering a specific button, or speaking an end word, etc. Of course, the client 110 can also determine that the current dialog interaction is ended in the case of triggering an end condition (for example, the digital assistant is in a standby state for more than a certain length of time, the time interval between the time when the client 110 last received the user speech 145 and the current time exceeds a certain length of time, etc.). The entire process from starting an interactive dialog to ending the interactive dialog can be considered as completing an interactive dialog.

[0055] By determining the dialog turn, whether it is a first post-wakeup interaction, and the question and answer state, the current process, link, or situation of the human-computer dialogue can be accurately determined, and the current dialog scene (i.e., the dialog scene corresponding to the question speech) can be accurately determined. It should be noted that the above examples of the dialog turn, the question and answer state, and whether it is a first post-wakeup interaction are only exemplary, and in actual application, the dialog turn, the question and answer state, and whether it is a first post-wakeup interaction can be flexibly determined according to actual needs.

[0056] In the embodiment of the present disclosure, as shown in the signaling flow 200, the client 110 can determine (208) a response strategy of the dialog scene based on the dialog scene. The client 110 can send (210) the response strategy of the dialog scene to the server 130, and the server 130 can receive (212) the response strategy of the dialog scene from the client 110.

[0057] Alternatively or additionally, the response strategy comprises at least one of a speech processing strategy for the question speech and a response processing strategy for the question speech. Illustratively, the speech processing strategy can comprise a strategy of performing speech recognition on the question speech, and the response processing strategy can comprise a strategy of generating or feeding back a response to the question speech. Here, the response can be broadly understood as including a response text, a response speech, and a prompt information for prompting the user to re-describe the question or informing the user that the question cannot be answered, and the like. On this basis, the response processing strategy can comprise a strategy of generating a response text, a strategy of generating a response speech, and a strategy of whether to generate a response, and the like.

[0058] In some embodiments, the speech processing strategy in the response strategy comprises at least a sensitivity of performing speech recognition (ASR) on the question speech. Generally, in a voice-based human-computer dialogue, after receiving the question speech, in order to be able to invoke a machine learning model supporting text input to determine the response corresponding to the question, it is necessary to identify the question text corresponding to the question speech through an ASR model. The ASR sensitivity can affect the ability of the ASR model to convert sound into text. Specifically, the ASR sensitivity can be positively correlated with the ability of the ASR model to convert sound into text. The greater the value of the ASR sensitivity, the higher the ability of the ASR model to convert sound into text. In this way, some sounds with lower energy and / or clarity can be recognized as text. The smaller the value of the ASR sensitivity, the lower the ability of the ASR model to convert sound into text. In this way, some sounds with lower energy and / or clarity can be excluded.

[0059] As shown in FIG. 3, the speech module 350 can comprise an audio module 351, which can perform an ASR function and / or a TTS function with the aid of an ASR model 371 and / or a TTS model 372 in the voice service 370. The ASR model 371 and / or the TTS model 372 are deployed on the server side 130, which can also be referred to as the ASR model 371 and / or the TTS model 372 being deployed on the cloud. Illustratively, the audio module 351 can send the user speech and the ASR sensitivity to the ASR model 371, and the ASR model 371 can perform speech recognition on the user speech based on the ASR sensitivity to obtain an ASR text. The audio module 351 can obtain the ASR text corresponding to the user speech (i.e., the question text converted from the question speech) from the ASR model 371.

[0060] In some embodiments, the response processing strategy in the response strategy at least includes a rejection threshold for the question text corresponding to the question voice, and the rejection threshold is positively correlated with the passing rate of the question text in the semantic detection process. In order to avoid providing the question text with incomplete semantics, incoherence, or illogicality to the question model 380, thereby avoiding occupying the resources of the question model 380, after obtaining the question text, the semantic detection (such as semantic integrity, coherence, logicality, and the like) can be performed on the question text. The rejection threshold can affect the strictness of the semantic detection, and in turn affect the passing rate of the semantic detection. Specifically, the greater the value of the rejection threshold, the looser the semantic detection, and the higher the passing rate of the semantic detection. The smaller the value of the rejection threshold, the stricter the semantic detection, and the lower the passing rate of the semantic detection.

[0061] Exemplarily, as shown in FIG. 3, the voice module 350 can further include a rejection module 352, which performs semantic detection (such as integrity, coherence, logicality, and the like) on the ASR text to obtain a detection result capable of indicating whether the ASR text passes the semantic detection. The rejection threshold can be positively correlated with the passing rate of the semantic detection performed by the rejection module 352 on the ASR text. Specifically, if the rejection threshold is higher, the rejection module 352 performs looser semantic detection on the ASR text, and the passing rate of the semantic detection is higher. In this way, the ASR text with lower semantic integrity or logical coherence can also be provided to the subsequent question and answer model for determining the response text. If the rejection threshold is lower, the rejection module 352 performs stricter semantic detection on the ASR text, and the passing rate of the semantic detection is lower. In this way, only the ASR text with higher semantic integrity or logical coherence can be provided to the subsequent question and answer model for determining the response text.

[0062] It can be understood that the specific content of the above-mentioned response strategy is only exemplary. In actual application, the response strategy is not limited to including the ASR sensitivity and the rejection threshold, and the specific content of the response strategy can be flexibly selected and configured according to actual needs. In addition, the response strategy corresponding to the specific dialogue scenario will be described later in combination with the specific dialogue scenario, which will not be described here, and can be referred to the subsequent description.

[0063] In the embodiments of the present disclosure, as shown in the signaling flow 200, the server 130 can generate (214) the question text based on the voice processing strategy and the question voice. The server 130 can send (216) the question text to the client 110, and the client 110 can receive (218) the question text from the server 130. The client 110 can present (220) the question text.

[0064] In some embodiments, the server 130 can utilize a first machine learning model to generate the answer text based on the question text. Illustratively, as shown in FIG. 3, the speech module 350 can send the ASR sensitivity and the user speech to the audio module 351. The audio module 351 can provide the ASR sensitivity and the user speech to the ASR model 371 (also referred to herein as “the first machine learning model”), which can perform speech recognition on the user speech based on the ASR sensitivity to generate the ASR text. The speech module 350 can send the obtained ASR text to the network interface 341 of the access layer 340. The access layer 340 can in turn send the obtained ASR text to the application speech module 311 of the client 110. Alternatively or additionally, the access layer 340 can send the ASR text to the client 110 via a long connection between the network interface 341 and the client 110. The application speech module 311 can send the ASR text to the application message module 312, which can present the ASR text on the interface 150 of the digital assistant.

[0065] In the embodiments of the present disclosure, as shown in the signaling flow 200, the server 130 can generate (222) the answer text based on the question text and the answer processing strategy. The server 130 can send (224) the answer text to the client 110, which can receive (226) the answer text from the server 130.

[0066] In some embodiments, the server 130 can perform a semantic detection process on the question text based on the rejection threshold to obtain a detection result indicating whether the question text passes the semantic detection. If the detection result indicates that the question text passes the semantic detection, the server 130 can utilize a second machine learning model to generate the answer text based on the question text.

[0067] Illustratively, as shown in FIG. 3, the speech module 350 can also provide the rejection threshold and the ASR text to the rejection module 352, which can perform semantic detection on the ASR text based on the rejection threshold to obtain a detection result. If the detection result indicates that the ASR text passes the semantic detection, the rejection module 352 can construct a sending message based on the ASR text and send the sending message to the question-answering module 360 of the server 130. The question-answering module 360 can construct a model input of the question-answering model 380 (also referred to herein as “the second machine learning model”) based on the sending message, provide the model input to the question-answering model 380, and utilize the question-answering model 380 to generate the answer text. The question-answering module 360 can send the answer text to the access layer 340, and the network interface 342 of the access layer 340 can utilize a second network connection to send the answer text to the application message module 312 of the client 110. If the detection result indicates that the ASR text does not pass the semantic detection, a prompt information can be fed back to the client 110 to prompt that the current question-answering does not pass the semantic detection.

[0068] In the embodiments of the present disclosure, as shown in the signaling flow 200, the server 130 can generate (228) the response speech based on the response text by using the third machine learning model in response to the response text being generated. The server 130 can send (230) the response speech to the client 110, and the client 110 can receive (232) the response speech from the server 130. Then, the client 110 can play the response speech and present the response text.

[0069] Exemplarily, as shown in FIG. 3, the question and answer module 360 can send the response text to the speech module 350, and the speech module 350 can provide the response text to the audio module 351, and trigger the TTS model 372 (also referred to as the “third machine learning model” herein sometimes) to generate the response speech corresponding to the response text by using the audio module 351. The response speech here is the speech converted from the response text, which can also be referred to as the TTS speech.

[0070] After receiving the TTS speech, the speech module 350 can send the TTS speech to the network interface 341 of the access layer 340, and send the TTS speech to the application speech module 311 of the client 110 via the first network connection. It can be understood that, in actual application, the server 130 can feed back the response text to the client 110 according to the human-computer dialogue mode of the digital assistant, or feed back the response speech to the client 110, or can also feed back the response text and the response speech to the client 110 respectively. For example, in the case where the digital assistant is configured to have a text dialogue with the user 140, the server 130 can feed back the response text to the client 110. In the case where the digital assistant is configured to have a speech dialogue with the user 140, the server 130 can feed back the TTS speech to the client 110. In the case where the digital assistant is configured to have a text and speech dialogue with the user 140, the server 130 can feed back the response text and the TTS speech to the client 110 respectively. The client 110 can play the TTS speech by using the application speech module 311, and present the response text on the user interface 150 synchronously with the playing of the TTS speech by using the application message module 312.

[0071] It should be noted that, in the above examples, the server 130 sends the question text, the response speech and the response text to the client in one dialogue round. However, in actual application, the server 130 can only feed back the question text to the client 110 in one dialogue round. For example, in the case where the semantic detection fails, the server 130 can only feed back the question text to the client 110, without feeding back the response to the client 110. According to the actual needs of the digital assistant, the server 130 can only feed back the response (such as the response speech and / or the response text) to the client 110, without feeding back the question text to the client 110.

[0072] It should be noted that in the above examples, the determination of the response strategy for the dialogue scenario is described by taking the client 110 as an example. However, in actual application, the server 130 can also determine the response strategy for the dialogue scenario. For example, after determining the dialogue scenario corresponding to the question voice, the client 110 can send an indication of the dialogue scenario to the server 130. The server 130 can determine the response strategy for the dialogue scenario in response to receiving the indication of the dialogue scenario. The process of determining the response strategy for the dialogue scenario by the server 130 is similar to the process of determining the response strategy for the dialogue scenario by the client 110, and will not be described here.

[0073] In order to better understand the corresponding relationship between the dialogue scenario and the response strategy, the specific content of the response strategy taken by each specific dialogue scenario will be described by way of example in combination with some specific dialogue scenarios. However, it should not be understood as being limited to the dialogue scenarios shown below, nor should it be understood as being limited to the response strategies shown below. In actual application, the dialogue scenarios and the corresponding response strategies can be flexibly selected and configured according to actual needs.

[0074] In some embodiments, if the client 110 determines that the question-answering state is the first state (for example, the waiting-receiving state), the response processing strategy can include a first rejection threshold. If it is determined that the question-answering state is the second state (for example, the thinking state or the reply state), the response processing strategy can include a second rejection threshold lower than the first rejection threshold.

[0075] For example, it is assumed that the value range of the rejection threshold is 0-1. If the digital assistant of the human-computer dialogue currently does not have user voice 145 that needs to be responded to, the client 110 is in the waiting-receiving state to wait for receiving the voice in the environment. At this time, there is no problem of interrupting the dialogue. If the client 110 receives the user voice 145 of the user 140, the response processing strategy corresponding to the waiting-receiving state can include a first rejection threshold (for example, 0.7, 0.8, or 0.9, etc.). The rejection module 352 of the server 130 has a relatively high pass rate of performing semantic detection on the ASR text based on the first rejection threshold, thereby being able to improve the probability of the question-answering model 380 generating the response text to the ASR text, so as to improve the probability of the user 140 obtaining the response.

[0076] If the digital assistant is currently in the thinking state or the replying state, there is a problem of the previous round of conversation being interrupted at this time. If the client 110 receives the user speech 145, it can be determined that the speech processing strategy includes a second rejection threshold (e.g., 0.1, 0.2, or 0.3, etc.). The rejection module 352 of the server 130 performs semantic detection on the ASR text based on the second rejection threshold, and the pass rate of the semantic detection is relatively low. Only when the semantic completeness, fluency, etc. of the ASR text is relatively high, the rejection module 352 provides the ASR text to the question-answering module 360. In this way, in a noisy environment, if the user has already asked a question and the digital assistant is responding to the question, reducing the rejection threshold can reduce the probability of subsequent speech being provided to the question-answering model 380, and thus can reduce the probability of the user's question being interrupted, so as to solve the problem of frequent interruption of human-computer conversation in a noisy environment.

[0077] In some embodiments, the client 110 can determine that the speech processing strategy includes the first sensitivity and the response processing strategy includes the first rejection threshold if it is determined that the conversation turn is less than the threshold turn and the question-answering state is the first state. The client 110 can determine that the speech processing strategy includes the second sensitivity and the response processing strategy includes the second rejection threshold if it is determined that the conversation turn is less than the threshold turn and the question-answering state is the second state, the second sensitivity being lower than the first sensitivity, and the second rejection threshold being lower than the first rejection threshold.

[0078] Exemplarily, it is assumed that the value range of the ASR sensitivity is 0-1, and the value of the ASR sensitivity is positively correlated with the ability of the ASR model 371 to convert sound into text. The value range of the rejection threshold can be 0-1, and the value of the rejection threshold is positively correlated with the pass rate of the semantic detection performed by the rejection module 352.

[0079] If it is determined that the current conversation turn is less than the threshold turn (e.g., 4 times, 5 times, 6 times, etc.) and the digital assistant is in the waiting to receive state, there is no problem of interrupting the conversation at this time, and the user 140 can have a strong conversation intention. The ASR sensitivity can be increased (e.g., the ASR sensitivity is determined to be 0.65, 0.7, etc.) and the rejection threshold can be increased (e.g., the rejection threshold is determined to be 0.8, 0.85, 0.9, etc.), to increase the ability of the ASR model 371 to generate ASR text based on the user speech 145, and to increase the probability of the question-answering model 380 generating a response text to the ASR text, so as to increase the probability of the user 140 obtaining a response.

[0080] If the digital assistant is responding to a previous user speech 145 of the user 140 (i.e., the question-answering state is the thinking state or the reply state), and the current dialog turn is less than a threshold turn (e.g., 4 turns, 5 turns, 6 turns, etc.), there is a possibility that the previous turn of the dialog is interrupted. If the client 110 receives the user speech 145, the ASR sensitivity can be reduced (e.g., the ASR sensitivity is reduced to 0.5, 0.55, etc.) and the rejection threshold is reduced (e.g., the rejection threshold is reduced to 0.1, 0.15, or 0.2, etc.), reducing the ability of the ASR model 371 to recognize ASR text from the user speech 145 and increasing the strictness of the semantic detection performed by the rejection module 352, reducing the probability that the digital assistant generates a reply text to the ASR text, and in turn, reducing the probability that the previous turn of the dialog of the user 140 is interrupted.

[0081] In some embodiments, the client 110 can determine that the speech processing policy includes a first sensitivity and the reply processing policy includes a first rejection threshold if it is determined that the dialog turn is less than a threshold turn and the question-answering state is a first state. The client 110 can determine that the speech processing policy includes a second sensitivity and the reply processing policy includes the first rejection threshold if it is determined that the dialog turn is greater than or equal to the threshold turn and the question-answering state is the first state, the second sensitivity being lower than the first sensitivity.

[0082] Illustratively, assume that the ASR sensitivity ranges from 0 to 1 and the rejection threshold ranges from 0 to 1. If it is determined that the current dialog turn is less than a threshold turn (e.g., 3 turns, 4 turns, 5 turns, etc.) and the digital assistant is in the waiting-to-receive state, the ASR sensitivity can be increased (e.g., the ASR sensitivity is determined to be 0.65, 0.7, etc., such that the speech recognition ability of the ASR model 371 is higher than the first ability) and the rejection threshold is increased (e.g., the rejection threshold is determined to be 0.8, 0.85, 0.9, etc., such that the pass rate of the semantic detection performed by the rejection module 352 is greater than the first pass rate).

[0083] If it is determined that the current dialog turn is greater than the threshold turn and the digital assistant is in the waiting-to-receive state, the ASR sensitivity can be reduced (e.g., the ASR sensitivity is determined to be 0.5) and the rejection threshold is kept high. Generally, as the dialog turn increases, the user’s willingness to interact decreases, and reducing the ASR sensitivity can reduce the conversion of unnecessary sounds into ASR text, can improve the accuracy of the conversion of the user speech 145 into ASR text in a noisy environment, and in turn, can improve the accuracy of the reply and avoid replying to the interference sounds in the environment.

[0084] In some embodiments, the client 110 can determine that the speech processing policy includes the second sensitivity and the response processing policy includes the first rejection threshold if the client 110 determines that the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state. The client 110 can determine that the speech processing policy includes the second sensitivity and the response processing policy includes a second rejection threshold, which is lower than the first rejection threshold, if the client 110 determines that the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the second state.

[0085] Exemplarily, if the digital assistant is in the waiting for receiving state, the ASR sensitivity can be reduced (e.g., the ASR sensitivity is configured to make the speech recognition capability of the ASR model 371 lower than the second capability) and the value of the rejection threshold can be increased (e.g., the rejection threshold is configured to make the passing rate of the semantic detection performed by the rejection module 352 greater than the first passing rate) if the current dialogue turn is greater than the threshold turn (e.g., 5 times). If the digital assistant is in the thinking state or the replying state, the value of the ASR sensitivity can be kept at a lower level and the value of the rejection threshold can be reduced (e.g., the rejection threshold is configured to make the passing rate of the semantic detection performed by the rejection module 352 lower than the second passing rate). In this way, in a noisy environment, the accuracy of speech recognition can be improved, and the dialogue can be prevented from being frequently interrupted.

[0086] In some embodiments, the client 110 can determine whether the question speech contains an execution instruction indicating a target operation if the client 110 determines that the current dialogue interaction in which the question speech is located is a non-first wake-up post-interaction, the target operation belonging to a non-answer type of operation. The client 110 can determine that the response policy further includes an anti-interrupt identifier if the client 110 determines that the question speech contains the execution instruction and the question-answer state is the second state. The client 110 can send the response policy including the anti-interrupt identifier to the server 130. The server 130 can generate question text based on the question speech, and can provide the question text and the anti-interrupt identifier to the second machine learning model to prevent interrupting the response operation of the second machine learning model to the previous question text.

[0087] Exemplarily, in the case that the current dialogue interaction is a non-first wake-up post-interaction and the digital assistant is in the response state to the previous user speech 145 of the user 140, if the user 140 issues a user speech 145 containing an execution instruction of a non-answer type of operation (e.g., turning off Bluetooth, increasing screen brightness, reducing volume, etc.), the anti-interrupt identifier can be added in the response policy. The question-answer module 360 of the server 130 can provide the anti-interrupt identifier and the ASR text to the question-answer model 380, which can prevent interrupting the response operation of the question-answer model 380 to the previous ASR text, and can also make the question-answer model 380 aware of the current dialogue content, thereby avoiding unnecessary interruption of the current dialogue.

[0088] In some embodiments, the client 110 can perform a target operation based on the execution instruction. The client 110 can also send the execution instruction to the server 130, instructing the server 130 to perform the target operation. For example, the client 110 can turn off Bluetooth, increase screen brightness, or reduce volume, etc. Also for example, the client 110 can send the execution instruction to the server 130, and the server 130 can perform a non-answering type operation (e.g., interface setting, parameter modification, data saving, etc.) based on the execution instruction.

[0089] To sum up, according to the embodiments of the present disclosure, in the application scenario of human-computer dialogue by voice, the response strategy can be determined based on the dialogue scenario, and the question voice can be responded based on the response strategy, which can avoid the problem that the human-computer dialogue is frequently interrupted in a noisy environment.

[0090] FIG. 4 shows a flowchart of a process 400 for question and answer processing, according to some embodiments of the present disclosure. The process 400 can be implemented at the client 110.

[0091] At block 410, the client 110 sends the question voice to the server in response to receiving the question voice of the user.

[0092] At block 420, the client 110 determines a dialogue scenario corresponding to the question voice.

[0093] At block 430, the client 110 sends an indication of the dialogue scenario or a response strategy of the dialogue scenario to the server, the response strategy including at least one of: a voice processing strategy for the question voice, an answering processing strategy for the question voice.

[0094] At block 440, the client 110 receives a question text and / or an answer for the question voice from the server, wherein the question text is determined based on the voice processing strategy and the question voice, and / or the answer is determined based on the question text and the answering processing strategy.

[0095] In some embodiments, the process 400 is further configured to determine a response strategy of the dialogue scenario based on the dialogue scenario, and send the response strategy of the dialogue scenario to the server.

[0096] In some embodiments, the voice processing strategy in the response strategy includes at least a sensitivity of performing voice recognition on the question voice; and / or the answering processing strategy in the response strategy includes at least a rejection threshold of the question text corresponding to the question voice, the rejection threshold being positively correlated with a passing rate of the question text in a semantic detection process.

[0097] In some embodiments, the process 400 is further configured to determine the dialogue scenario corresponding to the question voice based on at least one of: a comparison of a dialogue turn of the question voice in a current dialogue interaction with a threshold turn; whether the current dialogue interaction in which the question voice is located is a first wake-up post interaction or a non-first wake-up post interaction; a question and answer state corresponding to the dialogue turn in which the question voice is located, the question and answer state comprising a first state of waiting for receiving a user question or a second state of responding to the user question.

[0098] In some embodiments, the process 400 is further configured to determine that the response processing strategy comprises a first rejection threshold if the question and answer state is the first state, and determine that the response processing strategy comprises a second rejection threshold lower than the first rejection threshold if the question and answer state is the second state.

[0099] In some embodiments, the process 400 is further configured to determine that the voice processing strategy comprises a first sensitivity and the response processing strategy comprises a first rejection threshold if the dialogue turn is less than the threshold turn and the question and answer state is the first state, and determine that the voice processing strategy comprises a second sensitivity and the response processing strategy comprises a second rejection threshold if the dialogue turn is less than the threshold turn and the question and answer state is the second state, the second sensitivity being lower than the first sensitivity, and the second rejection threshold being lower than the first rejection threshold.

[0100] In some embodiments, the process 400 is further configured to determine that the voice processing strategy comprises a first sensitivity and the response processing strategy comprises a first rejection threshold if the dialogue turn is less than the threshold turn and the question and answer state is the first state, and determine that the voice processing strategy comprises a second sensitivity and the response processing strategy comprises the first rejection threshold if the dialogue turn is greater than or equal to the threshold turn and the question and answer state is the first state, the second sensitivity being lower than the first sensitivity.

[0101] In some embodiments, the process 400 is further configured to determine that the voice processing strategy comprises a second sensitivity and the response processing strategy comprises a first rejection threshold if the dialogue turn is greater than or equal to the threshold turn and the question and answer state is the first state, and determine that the voice processing strategy comprises the second sensitivity and the response processing strategy second rejection threshold if the dialogue turn is greater than or equal to the threshold turn and the question and answer state is the second state, the second rejection threshold being lower than the first rejection threshold.

[0102] In some embodiments, the process 400 is further configured to determine whether the question voice contains an execution instruction indicating a target operation if the current dialogue interaction in which the question voice is located is a non-first wake-up post interaction, the target operation belonging to a non-response type of operation, and determine that the response strategy further comprises an anti-interrupt identifier indicating preventing the server from interrupting a response operation to a previous question voice of the user if the question voice contains the execution instruction and the question and answer state is the second state.

[0103] In some embodiments, the process 400 is further configured to: at the client, perform a target operation based on the execution instruction.

[0104] FIG. 5 illustrates a flowchart of a process 500 for question and answer processing, according to some embodiments of the present disclosure. The process 500 can be implemented at the server 110.

[0105] At block 510, the server 130 receives, from the client, a user’s question speech.

[0106] At block 520, the server 130 receives, from the client, an indication of a dialogue scenario corresponding to the question speech or a response strategy for the dialogue scenario, the response strategy comprising at least one of: a speech processing strategy for the question speech, a response processing strategy for the question speech.

[0107] At block 530, the server 130 performs at least one of: obtaining a question text generated based on the speech processing strategy and the question speech, or determining a response to the question speech based on the question text and the response processing strategy, based on the response strategy.

[0108] At block 540, the server 130 sends the question text and / or the response to the client.

[0109] In some embodiments, the process 500 is further configured to: in response to receiving, from the client, the indication of the dialogue scenario corresponding to the question speech, determine the response strategy based on the dialogue scenario.

[0110] In some embodiments, the speech processing strategy in the response strategy comprises at least a sensitivity of performing speech recognition on the question speech; and / or the response processing strategy in the response strategy comprises at least a rejection threshold for a question text corresponding to the question speech, the rejection threshold being positively correlated with a passing rate of the question text in a semantic detection process.

[0111] In some embodiments, the process 500 is further configured to: recognize, based on the sensitivity, the question text from the question speech using a first machine learning model.

[0112] In some embodiments, the process 500 is further configured to: perform a semantic detection process on the question text based on the rejection threshold to obtain a detection result, the detection result indicating whether the question text passes the semantic detection; and generate, based on the question text, a response text using a second machine learning model if the detection result indicates that the question text passes the semantic detection.

[0113] In some embodiments, the process 500 is further configured to: in response to the response text being generated, generate a response speech based on the response text using a third machine learning model.

[0114] In some embodiments, the response policy further comprises an anti-interrupt identifier; the process 500 is further configured to provide the question-answer text and the anti-interrupt identifier to the second machine learning model to prevent interrupting the response operation of the second machine learning model to the previous question text.

[0115] In some embodiments, the process 500 is further configured to: if the question-answer state is the first state, determine that the response processing policy comprises a first rejection threshold; and if the question-answer state is the second state, determine that the response processing policy comprises a second rejection threshold lower than the first rejection threshold.

[0116] In some embodiments, the process 500 is further configured to: if the dialogue turn is less than the threshold turn and the question-answer state is the first state, determine that the speech processing policy comprises a first sensitivity and the response processing policy comprises a first rejection threshold; and if the dialogue turn is less than the threshold turn and the question-answer state is the second state, determine that the speech processing policy comprises a second sensitivity and the response processing policy comprises a second rejection threshold, the second sensitivity being lower than the first sensitivity, and the second rejection threshold being lower than the first rejection threshold.

[0117] In some embodiments, the process 500 is further configured to: if the dialogue turn is less than the threshold turn and the question-answer state is the first state, determine that the speech processing policy comprises a first sensitivity and the response processing policy comprises a first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing policy comprises a second sensitivity and the response processing policy comprises the first rejection threshold, the second sensitivity being lower than the first sensitivity.

[0118] In some embodiments, the process 500 is further configured to: if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing policy comprises a second sensitivity and the response processing policy comprises the first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the second state, determine that the speech processing policy comprises the second sensitivity and the response processing policy comprises a second rejection threshold, the second rejection threshold being lower than the first rejection threshold.

[0119] In some embodiments, the process 500 is further configured to: if the current dialogue interaction in which the question speech is located is a non-first wake-up interaction, determine whether the question speech contains an execution instruction indicating a target operation, the target operation belonging to a non-response type of operation; and if the question speech contains the execution instruction and the question-answer state is the second state, determine that the response policy further comprises an anti-interrupt identifier, the anti-interrupt identifier indicating preventing the server from interrupting a response operation to a previous question speech of the user.

[0120] In some embodiments, the process 500 is further configured to: at the server 130, execute the target operation based on the execution instruction.

[0121] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 6 shows an exemplary structural block diagram of an apparatus 600 for question and answer processing according to some embodiments of the present disclosure. The apparatus 600 can be implemented as or included in the client 110. Various modules / components in the apparatus 600 can be implemented by hardware, software, firmware, or any combination thereof.

[0122] As shown in FIG. 6, the apparatus 600 includes a question voice sending module 610, a dialogue scenario determining module 620, an indication strategy sending module 630, and an answer receiving module 640. The question voice sending module 610 is configured to send, at the client, a question voice to a server in response to receiving the question voice of a user. The dialogue scenario determining module 620 is configured to determine a dialogue scenario corresponding to the question voice. The indication strategy sending module 630 is configured to send, to the server, an indication of the dialogue scenario or a response strategy of the dialogue scenario, the response strategy including at least one of a voice processing strategy for the question voice, an answer processing strategy for the question voice. The answer receiving module 640 is configured to receive, from the server, a question text and / or an answer for the question voice, wherein the question text is determined based on the voice processing strategy and the question voice, and / or the answer is determined based on the question text and the answer processing strategy.

[0123] In some embodiments, the apparatus 600 further includes a response strategy determining module configured to determine, based on the dialogue scenario, a response strategy of the dialogue scenario; and the indication strategy sending module 630 is further configured to send, to the server, the response strategy of the dialogue scenario.

[0124] In some embodiments, the voice processing strategy in the response strategy at least includes a sensitivity of performing voice recognition on the question voice; and / or the answer processing strategy in the response strategy at least includes a rejection threshold of a question text corresponding to the question voice, the rejection threshold being positively correlated with a passing rate of the question text in a semantic detection process.

[0125] In some embodiments, the dialogue scenario determining module 620 is further configured to determine the dialogue scenario corresponding to the question voice based on at least one of: a comparison of a dialogue turn of the question voice in a current dialogue interaction with a threshold turn; whether a current dialogue interaction in which the question voice is located is a first wake-up after interaction or a non-first wake-up after interaction; and a question and answer state corresponding to a dialogue turn in which the question voice is located, the question and answer state including a first state of waiting for receiving a user question or a second state of responding to a user question.

[0126] In some embodiments, the response strategy determination module is further configured to: if the question-answer state is the first state, determine that the response processing strategy comprises a first rejection threshold; and if the question-answer state is the second state, determine that the response processing strategy comprises a second rejection threshold lower than the first rejection threshold.

[0127] In some embodiments, the response strategy determination module is further configured to: if the dialogue turn is less than the threshold turn and the question-answer state is the first state, determine that the speech processing strategy comprises a first sensitivity and the response processing strategy comprises a first rejection threshold; and if the dialogue turn is less than the threshold turn and the question-answer state is the second state, determine that the speech processing strategy comprises a second sensitivity and the response processing strategy comprises a second rejection threshold, the second sensitivity being lower than the first sensitivity, the second rejection threshold being lower than the first rejection threshold.

[0128] In some embodiments, the response strategy determination module is further configured to: if the dialogue turn is less than the threshold turn and the question-answer state is the first state, determine that the speech processing strategy comprises a first sensitivity and the response processing strategy comprises a first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing strategy comprises a second sensitivity and the response processing strategy comprises the first rejection threshold, the second sensitivity being lower than the first sensitivity.

[0129] In some embodiments, the response strategy determination module is further configured to: if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing strategy comprises a second sensitivity and the response processing strategy comprises the first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the second state, determine that the speech processing strategy comprises the second sensitivity and the response processing strategy comprises a second rejection threshold, the second rejection threshold being lower than the first rejection threshold.

[0130] In some embodiments, the response strategy determination module is further configured to: if the current dialogue interaction in which the question speech is located is a non-first wake-up interaction, determine whether the question speech contains an execution instruction indicating a target operation, the target operation belonging to a non-answer type of operation; and if the question speech contains the execution instruction and the question-answer state is the second state, determine that the response strategy further comprises an anti-interrupt identifier, the anti-interrupt identifier indicating that the server is prevented from interrupting a response operation to a previous question speech of the user.

[0131] In some embodiments, the apparatus 600 further comprises an operation execution module configured to execute the target operation at the client based on the execution instruction.

[0132] It should be understood that one or more steps in the above methods can be performed by a suitable electronic device or combination of electronic devices. Such an electronic device or combination of electronic devices may, for example, comprise the client 110 in FIG. 1.

[0133] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 7 shows an exemplary structural block diagram of an apparatus 700 for question and answer processing according to some embodiments of the present disclosure. The apparatus 700 can be implemented as or included in the server 130. Various modules / components in the apparatus 700 can be implemented by hardware, software, firmware, or any combination thereof.

[0134] As shown in FIG. 7, the apparatus 700 includes a question voice receiving module 710, an indication policy receiving module 720, a response policy executing module 730, and an answer sending module 740. The question voice receiving module 710 is configured to receive, at the server, a question voice of a user from a client. The indication policy receiving module 720 is configured to receive, from the client, an indication of a dialogue scenario corresponding to the question voice or a response policy of the dialogue scenario, the response policy including at least one of: a voice processing policy for the question voice, an answer processing policy for the question voice. The response policy executing module 730 is configured to execute, based on the response policy, at least one of: obtaining a question text generated based on the voice processing policy and the question voice, or determining an answer for the question voice based on the question text and the answer processing policy. The answer sending module 740 is configured to send the question text and / or the answer to the client.

[0135] In some embodiments, the apparatus 700 can further include a response policy determining module configured to determine, in response to receiving the indication of the dialogue scenario corresponding to the question voice from the client, the response policy based on the dialogue scenario.

[0136] In some embodiments, the voice processing policy in the response policy includes at least a sensitivity of performing voice recognition on the question voice; and / or the answer processing policy in the response policy includes at least a rejection threshold of the question text corresponding to the question voice, the rejection threshold being positively correlated with a passing rate of the question text in a semantic detection process.

[0137] In some embodiments, the response policy executing module 730 is further configured to recognize, based on the sensitivity, the question text from the question voice by using a first machine learning model.

[0138] In some embodiments, the response policy executing module 730 is further configured to perform, based on the rejection threshold, a semantic detection process on the question text to obtain a detection result, the detection result indicating whether the question text passes the semantic detection; and if the detection result indicates that the question text passes the semantic detection, generate, based on the question text, an answer text by using a second machine learning model.

[0139] In some embodiments, the response policy execution module 730 is further configured to, in response to the answer text being generated, generate, with a third machine learning model, answer speech based on the answer text.

[0140] In some embodiments, the response policy further includes an anti-interrupt identification; the apparatus 700 can further include an identification providing module configured to provide the question-answer text and the anti-interrupt identification to the second machine learning model to prevent interrupting the answer operation of the second machine learning model on the previous question text.

[0141] In some embodiments, the response policy determination module is further configured to: if the question-answer state is the first state, determine that the answer processing policy includes a first rejection threshold; and if the question-answer state is the second state, determine that the answer processing policy includes a second rejection threshold lower than the first rejection threshold.

[0142] In some embodiments, the response policy determination module is further configured to: if the dialogue turn is less than a threshold turn and the question-answer state is the first state, determine that the speech processing policy includes a first sensitivity and the answer processing policy includes a first rejection threshold; and if the dialogue turn is less than the threshold turn and the question-answer state is the second state, determine that the speech processing policy includes a second sensitivity and the answer processing policy includes a second rejection threshold, the second sensitivity being lower than the first sensitivity, and the second rejection threshold being lower than the first rejection threshold.

[0143] In some embodiments, the response policy determination module is further configured to: if the dialogue turn is less than a threshold turn and the question-answer state is the first state, determine that the speech processing policy includes a first sensitivity and the answer processing policy includes a first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing policy includes a second sensitivity and the answer processing policy includes the first rejection threshold, the second sensitivity being lower than the first sensitivity.

[0144] In some embodiments, the response policy determination module is further configured to: if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the first state, determine that the speech processing policy includes a second sensitivity and the answer processing policy includes the first rejection threshold; and if the dialogue turn is greater than or equal to the threshold turn and the question-answer state is the second state, determine that the speech processing policy includes the second sensitivity and the answer processing policy includes a second rejection threshold, the second rejection threshold being lower than the first rejection threshold.

[0145] In some embodiments, the response strategy determination module is further configured to: if the current dialogue interaction in which the question voice is located is a non-first wake-up interaction, determine whether the question voice contains an execution instruction indicating a target operation, the target operation belonging to a non-answer type operation; and if the question voice contains the execution instruction and the question and answer state is the second state, determine that the response strategy further comprises an anti-interrupt identifier, the anti-interrupt identifier indicating that the server is prevented from interrupting a response operation to a previous question voice of the user.

[0146] In some embodiments, the apparatus 700 further comprises an operation execution module configured to execute the target operation based on the execution instruction at the server 130.

[0147] The units and / or modules included in the apparatus 700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, e.g., machine executable instructions stored on a storage medium. In addition to or alternatively, some or all of the units and / or modules in the apparatus 700 can be implemented at least partially by one or more hardware logic components. As an example and not by way of limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0148] It should be understood that one or more steps in the above methods can be performed by an appropriate electronic device or combination of electronic devices. Such electronic devices or combinations of electronic devices can include, for example, the server 130 in FIG. 1.

[0149] FIG. 8 shows a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure can be implemented. It should be understood that the electronic device 800 shown in FIG. 8 is merely an example and should not be construed to limit the functionality and scope of the embodiments described herein. The electronic device 800 shown in FIG. 8 can be used to implement the client 110 or the server 130 of FIG. 1, and the electronic device 800 can also be used to implement the apparatus 600 of FIG. 6 or the apparatus 700 of FIG. 7.

[0150] As shown in FIG. 8, electronic device 800 is in the form of a general-purpose electronic device. Components of electronic device 800 can include, but are not limited to, one or more processors or processing units 810, memory 820, storage 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit(s) 810 can be actual or virtual processors and capable of executing various processing in accordance with programs stored in memory 820. In a multi-processing system, multiple processing units execute computer-executable instructions in parallel to improve the processing power of electronic device 800.

[0151] Electronic device 800 typically includes a plurality of computer storage media. Such media can be removable and / or non-removable, and can include volatile and / or nonvolatile media. Memory 820 can be volatile (such as, for example, registers, cache, random access memory (RAM)), non-volatile (such as, for example, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage 830 can be removable or non-removable and can include machine-readable media, such as, for example, flash drives, disks, or any other media capable of storing information and / or data and accessible by electronic device 800.

[0152] Electronic device 800 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 8, a disk drive or other computer-readable media drive can be provided for reading from or writing to a removable, non-removable, volatile, or non-volatile media slot (such as a "floppy" disk). In such cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. Memory 820 can include a computer program product 825 having one or more program modules configured to carry out the various methods or actions of the various embodiments of the present disclosure.

[0153] Communication unit(s) 840 enable communication with other electronic devices via communication media. Additionally, functionality of components of electronic device 800 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, electronic device 800 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes.

[0154] Input device 850 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. Output device 860 can be one or more output devices, such as a display, a speaker, a printer, etc. Electronic device 800 can also communicate with one or more external devices (not shown), such as a storage device, a display device, etc., through communication unit 840, as desired, in order to communicate with a user in order to interact with electronic device 800, or to communicate with any device (e.g., a network card, a modem, etc.) that enables electronic device 800 to communicate with one or more other electronic devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0155] According to an example implementation of the present disclosure, a computer readable storage medium is provided having computer executable instructions stored thereon, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0156] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0157] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0158] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0159] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (media) having instances of the software embodied thereon, such as computer software, firmware, wireless application protocol (WAP), middleware or microcode. For example, a computer program product can be a floppy disk, a CD-ROM, a DVD, a Blu-ray Disc™, a flash drive, a memory stick, a magnetic tape, or a hard disk drive. The machine-readable medium can be a single medium, or multiple media, of the same or different type. The computer program product can be one or more computer program components embodied in medium and / or transmission signals. The computer program product can have one or more computer program components embodied in medium and / or transmission signals.

[0160] The implementations of the disclosure have been described above with the intent to be illustrative rather than limiting. Although the implementations of the disclosure have been described with regard to one or more implementations, it will be recognized that a variety of modifications and changes can be made to these implementations without departing from the broader spirit and scope of the implementations as set forth in the preceding disclosure. For example, certain aspects of the implementations can be performed using hardware, software, and / or firmware, or any combination thereof. The above-described implementations should therefore be regarded as merely illustrative, and not as narrowing the scope of the disclosure, which is defined by the appended claims and their equivalents.

Claims

1. A method for question-answering processing, comprising: At the client end, in response to receiving the user's question voice, the client sends the question voice to the server. Determine the dialogue scenario corresponding to the question's voice; Send an instruction for the dialogue scenario or a response strategy for the dialogue scenario to the server. The response strategy includes at least one of the following: a voice processing strategy for the question voice and a response processing strategy for the question voice. as well as The server receives a problem text and / or a response to the problem voice, wherein the problem text is determined based on the voice processing strategy and the problem voice, and / or the response is determined based on the problem text and the response processing strategy.

2. The method according to claim 1, further comprising: Based on the dialogue scenario, determine the response strategy for the dialogue scenario, and Sending the instruction for the dialogue scenario or the response strategy for the dialogue scenario to the server includes: sending the response strategy for the dialogue scenario to the server.

3. The method according to claim 1 or 2, wherein the speech processing strategy in the response strategy includes at least a sensitivity for performing speech recognition on the question speech; and / or The response processing strategy in the response strategy includes at least a rejection threshold for the question text corresponding to the question speech, and the rejection threshold is positively correlated with the pass rate of the question text in the semantic detection process.

4. The method according to claim 3, wherein determining the dialogue scenario corresponding to the question voice includes: The dialogue scenario corresponding to the question voice is determined based on at least one of the following: The comparison between the question speech's dialogue turn in the current dialogue interaction and the threshold turn; The current dialogue interaction in which the question voice is located is either an interaction after the first wake-up or an interaction after a non-first wake-up. The question-and-answer state corresponding to the dialogue turn in which the question voice is located includes a first state of waiting to receive user questions or a second state of responding to user questions.

5. The method of claim 4, wherein determining the response strategy comprises: If the question-and-answer state is the first state, then the response processing strategy is determined to include a first rejection threshold; as well as If the question-and-answer state is the second state, then the response processing strategy is determined to include a second rejection threshold lower than the first rejection threshold.

6. The method of claim 4, wherein determining the response strategy comprises: If the number of dialogue rounds is less than the threshold number of rounds and the question-and-answer state is the first state, then it is determined that the voice processing strategy includes a first sensitivity and the response processing strategy includes a first rejection threshold. as well as If the number of dialogue rounds is less than the threshold number of rounds and the question-and-answer state is the second state, then it is determined that the voice processing strategy includes a second sensitivity and the response processing strategy includes a second rejection threshold, wherein the second sensitivity is lower than the first sensitivity and the second rejection threshold is lower than the first rejection threshold.

7. The method of claim 4, wherein determining the response strategy comprises: If the number of dialogue rounds is less than the threshold number of rounds and the question-and-answer state is a first state, then the voice processing strategy is determined to include a first sensitivity and the response processing strategy includes a first rejection threshold. as well as If the number of dialogue rounds is greater than or equal to the threshold number of rounds and the question-and-answer state is the first state, then it is determined that the voice processing strategy includes a second sensitivity and the response processing strategy includes the first rejection threshold, wherein the second sensitivity is lower than the first sensitivity.

8. The method of claim 4, wherein determining the response strategy comprises: If the number of dialogue rounds is greater than or equal to the threshold number of rounds and the question-and-answer state is a first state, then it is determined that the voice processing strategy includes a second sensitivity and the response processing strategy includes a first rejection threshold. as well as If the number of dialogue rounds is greater than or equal to the threshold number of rounds and the question-and-answer state is the second state, then the voice processing strategy is determined to include the second sensitivity and the response processing strategy includes a second rejection threshold, wherein the second rejection threshold is lower than the first rejection threshold.

9. The method of claim 4, wherein determining the response strategy comprises: If the current dialogue interaction in which the question voice is located is not an interaction after the first wake-up, then determine whether the question voice contains an instruction to execute a target operation, and the target operation belongs to the non-response type operation; as well as If the question voice contains the execution instruction and the question-and-answer state is the second state, then it is determined that the response strategy also includes an anti-interruption flag, which indicates that the server should be prevented from interrupting the response operation to the user's previous question voice.

10. The method of claim 9, further comprising: At the client end, the target operation is performed based on the execution instruction.

11. A method for question-answering processing, comprising: On the server side, the system receives user questions via voice from the client. The client receives an instruction for a dialogue scenario corresponding to the problematic voice or a response strategy for the dialogue scenario, wherein the response strategy includes at least one of the following: a voice processing strategy for the problematic voice and a response processing strategy for the problematic voice; Based on the aforementioned response strategy, perform at least one of the following: Obtain the question text generated based on the speech processing strategy and the question speech, or Based on the question text and the response processing strategy, determine the response to the question voice. Send the question text and / or the response to the client.

12. The method of claim 11, further comprising: In response to receiving an indication from the client of a dialogue scenario corresponding to the question voice, the response strategy is determined based on the dialogue scenario.

13. The method of claim 11, wherein the speech processing strategy in the response strategy includes at least a sensitivity for performing speech recognition on the question speech; and / or The response processing strategy in the response strategy includes at least a rejection threshold for the question text corresponding to the question speech, and the rejection threshold is positively correlated with the pass rate of the question text in the semantic detection process.

14. The method of claim 13, wherein obtaining the question text comprises: The first machine learning model is used to identify the question text from the question speech based on the sensitivity.

15. The method of claim 13, wherein determining a response to the question voice comprises: A semantic detection process is performed on the question text based on the rejection threshold to obtain a detection result, which indicates whether the question text passes the semantic detection. as well as If the detection result indicates that the question text passes semantic detection, then a response text is generated based on the question text using a second machine learning model.

16. The method of claim 15, wherein determining a response to the question voice further comprises: In response to the generation of the response text, a third machine learning model is used to generate a response speech based on the response text.

17. The method of claim 15, wherein the response strategy further includes an anti-interruption flag; the method further includes: The question-and-answer text and the anti-interruption flag are provided to the second machine learning model to prevent interruption of the second machine learning model's response to the previous question text.

18. An apparatus for question-and-answer processing, comprising: The question voice sending module is configured to, on the client side, send the question voice to the server in response to receiving the user's question voice. The dialogue scenario determination module is configured to determine the dialogue scenario corresponding to the question voice. The instruction policy sending module is configured to send an instruction for the dialogue scenario or a response policy for the dialogue scenario to the server. The response policy includes at least one of the following: a voice processing policy for the question voice and a response processing policy for the question voice. as well as The response receiving module is configured to receive a problem text and / or a response to the problem voice from the server, wherein the problem text is determined based on the voice processing strategy and the problem voice, and / or the response is determined based on the problem text and the response processing strategy.

19. An apparatus for question-and-answer processing, comprising: The question voice receiving module is configured to receive the user's question voice from the client on the server side; The instruction policy receiving module is configured to receive from the client an instruction for a dialogue scenario corresponding to the question voice or a response policy for the dialogue scenario, wherein the response policy includes at least one of the following: a voice processing policy for the question voice and a response processing policy for the question voice; The response policy execution module is configured to execute at least one of the following based on the response policy: Obtain the question text generated based on the speech processing strategy and the question speech, or Based on the question text and the response processing strategy, determine the response to the question voice. The response sending module is configured to send the question text and / or the response to the client.

20. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, causing the electronic device to perform the method according to any one of claims 1 to 10 or 11 to 17.

21. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 10 or 11 to 17.

22. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 10 or 11 to 17.

Images