Method, apparatus, device, storage medium and program product for response processing

Abstract

This disclosure provides a method, apparatus, device, storage medium, and program product for response processing. The method includes: at a client device, in response to receiving a query request for a digital assistant during a session with the digital assistant, sending the query request to a server device via a predetermined first communication link; receiving a response message for the query request from the server device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link; and providing the response message as a response from the digital assistant to the query request. Thus, the client device and the server device can select an appropriate communication link from the plurality of communication links to transmit the response message, which can improve the quality and efficiency of data transmission during the response process, thereby improving the quality and efficiency of response processing.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, electronic devices, computer-readable storage media, and computer program products for response processing. Background Technology

[0002] With the development of information technology, various electronic devices can provide people with a variety of services in work and life. For example, application services can be deployed on client devices. Client devices or applications can provide users with digital assistant functions to assist users in using the client devices or applications. Users can complete a variety of operations through various interactions with the digital assistant. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for response processing is provided. The method is implemented at a client device and includes: in response to receiving a query request for a digital assistant during a session with the digital assistant, sending the query request to a server device via a predetermined first communication link; receiving a response message for the query request from the server device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link; and providing the response message as a reply from the digital assistant to the query request.

[0004] In a second aspect of this disclosure, a method for response processing is provided. The method is implemented at a server device and includes: receiving a query request for a digital assistant from a client device via a predetermined first communication link; determining a response message corresponding to the query request; and sending the response message to the client device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link.

[0005] In a third aspect of this disclosure, an apparatus for response processing is provided. The apparatus is implemented at a client device and includes: a query request sending module configured to send a query request to a server device via a predetermined first communication link in response to receiving a query request for a digital assistant during a session with the digital assistant; a response message receiving module configured to receive a response message for the query request from the server device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link; and a response message providing module configured to provide the response message as a reply from the digital assistant to the query request.

[0006] In a fourth aspect of this disclosure, an apparatus for response processing is provided. The apparatus is implemented at a server device and includes: a query request receiving module configured to receive a query request for a digital assistant from a client device via a predetermined first communication link; a response message determining module configured to determine a response message corresponding to the query request; and a response message sending module configured to send the response message to the client device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link.

[0007] In a fifth aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform a method according to a first or second aspect of this disclosure when executed by the at least one processing unit.

[0008] In a sixth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform a method according to a first or second aspect of this disclosure.

[0009] In a seventh aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first or second aspect.

[0010] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0012] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0013] Figure 2 A flowchart of a signaling flow for response processing according to some embodiments of the present disclosure is shown;

[0014] Figure 3 A flowchart of a method for response processing according to some embodiments of the present disclosure is shown;

[0015] Figure 4 A flowchart of a method for response processing according to other embodiments of the present disclosure is shown;

[0016] Figure 5 An exemplary structural block diagram of an apparatus for response processing according to some embodiments of the present disclosure is shown;

[0017] Figure 6 Exemplary structural block diagrams of an apparatus for response processing according to other embodiments of the present disclosure are shown; and

[0018] Figure 7 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0021] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0022] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0024] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0025] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0027] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0028] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0029] Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In this example environment 100, an application 112 and a digital assistant 114 are installed on a client device 110. A user 140 can interact with the application 112 via the client device 110 and / or an attached device of the client device 110. In some implementations, the application 112 may be authorized to capture speech via an audio capture device (e.g., a microphone) of the client device 110, capture images via an image capture device (e.g., a camera) of the client device 110, and so on.

[0030] In some embodiments, application 112 and digital assistant 114 may be downloaded and installed on client device 110. In some embodiments, application 112 and digital assistant 114 may also be accessed in other ways, such as through a web page.

[0031] In embodiments of this disclosure, application 112 can be any suitable application with response functionality, which may include, but is not limited to, one or more of the following: chat application components (also known as instant messaging application components), browser application components, planning application components, document application components, audio / video conferencing application components, email application components, task application components, calendar application components, goal and key results (OKR) application components, etc. It is understood that, although Figure 1 The image shows a single application component, but in reality, multiple application components can be installed on the client device 110. In some embodiments, application 112 may include a multi-functional collaboration platform, such as an office collaboration platform (also known as an office suite), which can provide integration of various types of business components to facilitate people's office work, communication, and other activities. In a multi-functional collaboration platform, people can launch different business components as needed to complete corresponding information processing, sharing, communication, etc.

[0032] In some embodiments, the digital assistant 114 may be provided by a separate application business component, or it may be integrated into an application 112 capable of providing content entities. The application business component providing the client interface for the digital assistant may correspond to a single-function application business component or a multi-functional collaboration platform, such as an office suite or other collaboration platform capable of integrating multiple components. It is understood that, similar to application business components, although... Figure 1 The image shows a single digital assistant, but there can actually be multiple digital assistants.

[0033] In some embodiments, the digital assistant 114 supports the use of plugins. Each plugin can provide one or more functions of the application. Such plugins include, but are not limited to, one or more of the following: search plugin, contact plugin, messaging plugin, document plugin, form plugin, email plugin, calendar plugin, schedule plugin, task plugin, etc.

[0034] Digital assistant 114 is a user's intelligent assistant, possessing intelligent dialogue and information processing capabilities. In embodiments of this disclosure, digital assistant 114 is used to interact with user 140 to assist user 140 in using terminal devices or applications. In some embodiments, multiple interaction modes between user 140 and digital assistant 114 can be provided, and users can flexibly switch between these modes. When a certain interaction mode is triggered, a corresponding interaction area is presented to facilitate interaction between user 140 and digital assistant 114. The interaction methods between user 140 and digital assistant 114 differ under different interaction modes, thus flexibly adapting to the interaction needs of different application scenarios.

[0035] In environment 100, in response to the launch of application 112, client device 110 may present an interface 150 of application 112 and / or digital assistant 114. Interface 150 may, for example, include an interactive interface for application 112 and digital assistant 114. In some embodiments, interface 150 may present an interaction window between user 140 and digital assistant 114. In the interaction window, user 140 can converse with digital assistant 114 by inputting natural language, images, audio files, video files, web page files, etc., to instruct the digital assistant to assist in completing various tasks.

[0036] The interaction window between the digital assistant 114 and the user 140 may include a session window, such as a session window in an instant messaging application or an instant messaging module of a specific application. In the session window, the interaction between the digital assistant 114 and the user 140 may be presented in the form of session messages. Alternatively or additionally, the interaction window between the digital assistant 114 and the user 140 may also include other types of windows, such as a floating window, in which the user 140 can trigger the digital assistant 114 to perform corresponding operations by entering commands, selecting shortcuts, etc.

[0037] In some embodiments, the digital assistant 114 may support a conversation window interaction mode, also known as conversation mode. In this interaction mode, a conversation window is presented between the user 140 and the digital assistant 114, where the user 140 and the digital assistant 114 interact through conversation messages. In conversation mode, the digital assistant 114 can perform tasks based on the conversation messages in the conversation window. In the interaction window, the user 140 inputs interaction messages, and the digital assistant 114 responds to the user's input by providing a reply message. A conversation window with the digital assistant 114 can be opened by selecting the digital assistant 114. The conversation window may include interface elements for information interaction, such as input boxes, message lists, message bubbles, etc.

[0038] In some embodiments, a communication connection is established between the client device 110 and the server device 120. The communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections, etc., and the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, the client device 110 and the server device 120 can perform signaling interaction through their communication connection to provide services to the application 112 and / or the digital assistant 114.

[0039] like Figure 1 As shown, server device 120 can invoke machine learning model 130 to support task processing and / or query response functions of application 112 and / or digital assistant 114 based on the output of machine learning model 130. Machine learning model 130 may include one or more machine learning models, which, for ease of description, are collectively referred to as machine learning model 130 herein. Machine learning model 130 may be deployed on server device 120 or on other devices.

[0040] Machine learning model 130 can be based on any suitable model architecture, including but not limited to Transformer models, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep neural networks (DNNs), and so on. In some embodiments, machine learning model 130 can be based on a language model (LM). A language model, by learning from a large corpus, is capable of question answering. Machine learning model 130 can also be based on other suitable models.

[0041] It should be noted that if machine learning model 130 includes multiple machine learning models, these multiple machine learning models may have the same function, structure, purpose, etc., or they may be different. In some embodiments, where client device 110 or application 112 can provide voice processing services to user 140, machine learning model 130 may include at least several voice-related machine learning models, such as a machine learning model for performing speech synthesis TTS (which may be simply referred to as a TTS model), a machine learning model for performing speech recognition ASR (which may be simply referred to as an ASR model), and a machine learning model for performing question answering (which may be simply referred to as a question answering model). The input of the ASR model is speech, and the output is text. The input of the TTS model is text, and the output is the corresponding speech. The input of the question answering model is the question text, and the output is the corresponding response text.

[0042] Client device 110 can be any suitable type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, client device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0043] The server-side device 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The server-side device 120 may include, for example, computing systems / servers such as mainframes, edge computing nodes, and computing devices in a cloud environment, etc.

[0044] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0045] As mentioned earlier, client devices can deploy applications that provide services. These client devices or applications can offer digital assistant-like functionalities to assist users in using the device or application. Users can perform various operations through interactions with the digital assistant. For example, in providing a response service, the client device can receive query requests from users directed to the digital assistant. The client device can then forward the received query request to the server device via the communication link between the client device and the server device, and receive a response from the server device. The client device can then provide this response to the user.

[0046] It can be observed that the quality of the response service is affected by the communication link. The higher the reliability, stability, and data transmission speed of the communication link, the higher the quality of the response service. When the client device is relatively simple (e.g., a microcontroller unit (MCU), it may not always support communication protocols with high device requirements. Therefore, the goal is to utilize simpler communication protocols for data transmission while ensuring the quality of the response service.

[0047] In view of the above, according to embodiments of the present disclosure, an improved solution for response processing is provided. According to the solution of an embodiment of the present disclosure, at the client device, in response to receiving a query request for a digital assistant during a session with the digital assistant, the query request is sent to the server device via a predetermined first communication link; a response message for the query request is received via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link; and the response message is provided as the digital assistant's response to the query request. At the server device, a request message is received from the client device via the predetermined first communication link, the request message being generated based on the query request for the digital assistant, the query request corresponding to a plurality of request messages, each request message including at least a portion of the query request; in response to receiving the plurality of request messages, a response message corresponding to the query request is determined; and the response message is sent to the client device via a target communication link among the plurality of communication links.

[0048] In this way, the client device and the server device can select an appropriate communication link from multiple communication links to transmit the response message, which can improve the quality and efficiency of data transmission during the response process, thereby improving the quality and efficiency of response processing.

[0049] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0050] Figure 2 A flowchart of a signaling flow 200 for response processing according to some embodiments of the present disclosure is shown. For ease of discussion, refer to... Figure 1 To describe signaling flow 200. For example... Figure 2 As shown, signaling flow 200 relates to client device 110 and server device 120. In some embodiments, server device 120 may include a Message Queuing Telemetry Transport (MQTT) service 201, a background service 202, an Automatic Speech Recognition (ASR) model 203, a language model 204, and a Text-to-Speech (TTS) model 205. MQTT service 201 can provide communication support in the case of communication based on the MQTT protocol. Background service 202 can provide background support for services related to the digital assistant. ASR model 203, language model 204, and TTS model 205 can be invoked to determine the processing of query requests or responses to the digital assistant.

[0051] ASR model 203 can be configured to perform speech recognition on received speech to determine the corresponding text, and TTS model 205 can be configured to perform speech synthesis on received text to determine the corresponding speech. Language model 204 is configured to generate a response corresponding to a received question. ASR model 203, language model 204, and TTS model 205 can all be based on any suitable model architecture, including but not limited to Transformer models, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep neural networks (DNNs), etc.

[0052] Client device 110 can receive (211) query requests sent by the user to the digital assistant during a session between the user (e.g., user 140) and the digital assistant (e.g., digital assistant 114). Client device 110 can receive query requests in any suitable manner. For example, client device 110 can receive voice-type query requests input by the user via a microphone, text-type query requests input by the user via an input field, gesture-type query requests via a camera, and so on. This disclosure does not limit the manner in which query requests are received.

[0053] In response to receiving a query request, client device 110 may send the query request to server device 120 via a predetermined first communication link. The first communication link may, for example, include a communication link based on the User Datagram Protocol (UDP) (which may be referred to as a UDP link). UDP is a simple transport layer protocol that provides a connectionless communication service. UDP is a fast but unreliable transport protocol that allows the transmission of encapsulated IP packets without guaranteeing the order, integrity, or reliability of the packets.

[0054] In some embodiments, client device 110 may directly send the query request to server device 120 via a UDP link. In other embodiments, client device 110 may generate (212) at least one request message based on the query request according to a predetermined data structure corresponding to the UDP protocol. It is understood that each request message includes at least a portion of the query request. It is also understood that the query requests included in different request messages do not overlap. For example, if client device 110 generates four request messages based on the query request, then each request message may include 1 / 4 of the content of the query request.

[0055] Client device 110 can, for example, determine the amount of query request content included in each request message according to a predetermined length. As an example, if the query request is an audio query, the predetermined length can be a predetermined duration, and each request message includes audio of the predetermined duration of the query request. For example, if the query request is 30 seconds of audio and the predetermined duration is 5 seconds, each request message can include 5 seconds of audio from the query request. If the query request is a text query, the predetermined length can be a predetermined number of texts, and each request message includes the predetermined number of texts from the query request. For example, if the query request is 50 words of text and the predetermined number of texts is 10, each request message can include 10 texts from the query request.

[0056] Each request message may also include a session identifier. The session identifier may include any suitable identifier such as text, symbols, images, icons, etc., and may also be referred to as a session ID. Each request message may also include the content type of the query request, which includes at least voice and text types. As an example, the content type of the query request may also include any suitable type such as image or video. Each request message may also include the order in which the request message is among at least one request message. Each request message may also include whether the request message is the last request message among at least one request message. It is understood that each request message may also include one or more of the above-mentioned session identifier, query request content type, order, and whether it is the last request message; this disclosure does not limit this. Referring to Table 1, Table 1 shows an example of a request message generated according to a predetermined data structure:

[0057] Table 1

[0058]

[0059] The client device 110 can then send (213) the request message to the server device 120 via a UDP link. Specifically, the client device 110 can send the request message to the background service 202 in the server device 120 via a UDP link. In some embodiments, if a query request corresponds to multiple request messages, the client device 110 can send only one request message to the server device 120 each time, or it can send a group of request messages to the server device 120 each time, and each group of request messages can include at least one request message.

[0060] Server device 120 (specifically, for example, background service 202) can directly receive query requests from client device 110 via a UDP link. In some embodiments, background service 202 can also receive request messages from client device 110 via a UDP link. These request messages are generated based on query requests for the digital assistant, and each query request may correspond to at least one request message, each request message including at least a portion of the query request.

[0061] In the case where a query request corresponds to multiple request messages, the background service 202 may, for example, respond to the received target request message including content indicating that it is the last request message among the multiple request messages, determine that the target request message is the last request message among the multiple request messages, and then the background service 202 may, based on the target order of the target request message among the multiple request messages, determine that multiple request messages have been received in response to determining that other request messages in the order prior to the target order have been received. The background service 202 may, for example, parse the multiple request messages according to a predetermined data structure corresponding to the UDP protocol to determine the query request (214).

[0062] Background service 202 can then determine the response message corresponding to the query request. Background service 202 can determine the content type of the query request. For example, background service 202 can determine the content type of the query request while parsing multiple request messages according to the predetermined data structure corresponding to the UDP protocol. If the content type of the query request is voice, the query request can be referred to as a voice query.

[0063] The background service 202 can provide (215) the query speech to the ASR model 203. The ASR model 203 can perform speech recognition on the query speech to determine the query text corresponding to the query speech. The background service 202 can obtain (216) the query text from the ASR model 203. The background service 202 can then provide (217) the query text to the language model 204. The language model 204 can determine the response text corresponding to the query text based on the query text. The background service 202 can obtain (218) the response text from the language model 204. The background service 202 can then provide (220) the response text to the TTS model 205.

[0064] In some embodiments, language model 204 can generate response text based on query text, and provide all response text to background service 202 after all response text has been generated (i.e., background service 202 can obtain all response text at once). In other embodiments, language model 204 can also generate response text in a streaming manner based on query text. In this case, background service 202 can obtain response text in a streaming manner from language model 204. For example, background service 202 can obtain response text word by word from language model 204. Background service 202 can detect (219) whether the received text can form a sentence, and in response to determining that the received text can form a sentence, provide the received sentence to TTS model 205.

[0065] The background service 202 can use any appropriate method to detect whether the received text can form a sentence. For example, the background service 202 can determine that the received text can form a sentence in response to the detection of punctuation marks such as a period “.”, an exclamation mark “!”, an ellipsis “…”, or a question mark “?”. For instance, the background service 202 can continue to receive new text from the language model 204 and continuously detect whether the newly received text can form a sentence, even if a received sentence has been provided to the TTS model 205 and the response text has not yet been fully received.

[0066] The TTS model 205 can perform speech synthesis on the received response text (which can be the entire response text or a sentence of the response text) to determine the response speech corresponding to the response text. The background service 202 can obtain the response speech (221) from the TTS model 205.

[0067] For example, if the content type of the query request is text, it can be directly referred to as the query text. The backend service 202 can directly provide this query text to the language model 204 and obtain the response text from the language model 204. In this case, the backend service 202 can determine the response speech without relying on the TTS model 205, or it can still determine the response speech using the TTS model 205, depending on the user's settings for the response. The response speech or response text can be collectively referred to as the response message determined by the backend service using a machine learning model (specifically, a language model).

[0068] In some embodiments, the backend service 202 may determine (222) a target communication link for message sending from a plurality of communication links, including at least a first communication link (e.g., a UDP link). In some embodiments, the plurality of communication links may further include a Message Queuing Telemetry Transport (MQTT) link and / or a communication link corresponding to the Hypertext Transfer Protocol (HTTP) (which may be referred to as an HTTP link). Depending on the type of the determined target communication link (e.g., a UDP link, an HTTP link, or an MQTT link), the sending of the response message may include the operations corresponding to schemes 206, 207, or 208.

[0069] MQTT is a lightweight, publish / subscribe-based messaging protocol typically used for communication between Internet of Things (IoT) devices and mobile devices. The MQTT protocol is designed to provide reliable message delivery in low-bandwidth, high-latency, or unreliable network environments. HTTP is an application-layer protocol for transmitting hypertext over a network, defining the standard for requests and responses between clients and servers. HTTP is stateless, meaning each request is independent, and the server does not retain any information about previous requests. UDP links have lower latency than MQTT and HTTP links, and their reliability is also lower than that of MQTT and HTTP links.

[0070] Background service 202 may, for example, determine the networking capabilities of client device 110 and, based on the networking capabilities of client device 110, determine the target communication link for message transmission from multiple communication links. In some embodiments, if a received query request corresponds to multiple request messages, background service 202 may, in response to receiving these multiple request messages, determine the networking capabilities of client device 110 based on the order indicated by the multiple request messages and the receiving order of the multiple request messages. As an example, background service 202 may determine the matching of the order indicated by the multiple request messages and the receiving order of the multiple request messages. It is understood that if the order indicated by a request message is the same as the receiving order of the request message (e.g., the 6th request message is the 6th received), then it is determined that the order indicated by the request message matches the receiving order of the request message.

[0071] The background service 202 can compare the number of request messages whose order of matching indication does not match the receiving order with a predetermined number, and determine the networking capability of the client device 110 based on the comparison result. For example, the background service 202 can determine that the client device 110 has a strong networking capability if the number of request messages whose order of matching indication does not match the receiving order does not reach the predetermined number, and determine that the client device 110 has a poor networking capability if the number of request messages whose order of matching indication does not match the receiving order reaches the predetermined number.

[0072] For example, if multiple request messages include 10 request messages, and the predetermined number is 5, then the background service 202 can determine that the client device 110 has poor networking capabilities if there are 6 request messages out of 10 that are not ordered and are received in the correct order, and can determine that the client device 110 has strong networking capabilities if there are 3 request messages out of 10 that are not ordered and are received in the correct order.

[0073] Background service 202 can, for example, determine the capability level corresponding to the networking capability of client device 110. Background service 202 can, for example, pre-determine the correspondence between the networking capability and capability level of client device 110. For example, background service 202 can determine that when the networking capability is in the range [a1, b1), it is at capability level A; when the networking capability is in the range [a2, b2), it is at capability level B; when the networking capability is in the range [a3, b3), it is at capability level C, and so on. It is understood that there are no identical values ​​in different ranges. For example, there may be no identical values ​​in the range [a1, b1) and the range [a2, b2).

[0074] The background service 202 can respond to a client device 110's network capability being lower than a certain capability level (e.g., capability level A), determining that its network capability is poor and a communication link with higher reliability is needed to ensure data transmission quality. The background service 202 can then designate a second communication link as the target communication link, the second communication link having higher communication reliability than the first communication link. Alternatively, the background service 202 can respond to a client device 110's network capability being higher than a certain capability level (e.g., capability level B), designating the first communication link as the target communication link. These two capability levels (i.e., capability level A and capability level B) can be the same or different capability levels.

[0075] When these two capability levels are different (i.e., capability level A ≠ capability level B), capability level A can be lower than capability level B. That is, the backend service 202 can determine the second communication link as the target communication link in response to the client device 110's network capability being lower than the lower capability level A, and can determine the first communication link as the target communication link in response to the client device 110's network capability being higher than the higher capability level B.

[0076] The background service 202 can, for example, determine the target communication link for message sending from multiple communication links based on the link indication sent by the client device 110 to the server device 120 (e.g., the background service 202). This link indication can be provided to the background service 202 via an MQTT link. That is, the client device 110 can send the link indication to the background service 202 via an MQTT link.

[0077] Client device 110 may, in response to receiving a selection of a target communication link among multiple communication links (e.g., client device 110 may determine that a selection of a target communication link has been received in response to receiving a user's selection operation for a target communication link), provide a link indication to the background service 202 of server device 120, the link indication including an indication of the selected target communication link. In some embodiments, client device 110 may also, in response to determining that a link indication has been sent to server device 120, determine the communication link indicated by the link indication as the target communication link for message reception.

[0078] Client device 110 can also detect its own networking capabilities. If its networking capabilities are below a certain capability level (e.g., capability level C), and it determines that its networking capabilities are poor, a communication link with higher reliability is needed to ensure data transmission quality. Client device 110 can then designate a second communication link as the target communication link and provide a link indication to the backend service 202, indicating that the second communication link has higher reliability than the first communication link. Alternatively, if its networking capabilities are above a certain capability level (e.g., capability level D), client device 110 can designate the first communication link as the target communication link and provide a link indication to the backend service 202. These two capability levels (i.e., capability level C and capability level D) can be the same or different capability levels.

[0079] Similarly, when these two capability levels are different (i.e., capability level C ≠ capability level D), capability level C can be lower than capability level D. That is, client device 110 can provide a link indication indicating a second communication link to the backend service 202 in response to client device 110's networking capability being lower than the lower capability level C, and can provide a link indication indicating a first communication link to the backend service 202 in response to client device 110's networking capability being higher than the higher capability level D.

[0080] The background service 202 can then determine the target communication link for message transmission from multiple communication links based on the received link indication. For example, the background service 202 can determine the target communication link as a UDP link in response to a link indication including an indication of a UDP link, or it can determine the target communication link as an HTTP link in response to a link indication including an indication of an HTTP link, and so on.

[0081] It should be noted that in some embodiments, the server device 120 determines the target communication link based on the received link indication received within a predetermined time period after receiving the query request / request message. Correspondingly, the client device 110 may also determine the communication link indicated by the link indication as the target communication link for message reception in response to determining that the link indication was sent to the server device 120 within a predetermined time period after sending the query request / request message. The predetermined time period can be determined based on the time difference between the server device 120 receiving the query request and sending a response message for that query request. For example, if the server device 120 receives the query request at time A and sends a response message at time B, the predetermined time period can be any appropriate time period shorter than time A - time B. The predetermined time period can also be determined based on predetermined settings.

[0082] As an example, the predetermined time period can be 5 milliseconds. If the client device 110 sends a link indication to the server device 120 within 5 milliseconds after sending a query request, it can determine the communication link indicated by the link indication as the target communication link for message reception. The server device 120 can also determine the target communication link based on the link indication if it receives the link indication within 5 milliseconds after receiving the query request. The server device 120 can also ignore the link indication if it receives the link indication at 7 milliseconds after receiving the query request, or determine the communication link for sending the next response message as the communication link indicated by the link indication. It can be understood that if the link indication is sent before or together with the query request, the communication link for sending the response message for the query request can be determined based on the link indication. Thus, the client device 110 / server device 120 can determine the communication link based on the link indication successfully sent / received within the predetermined time period.

[0083] The background service 202 can, for example, determine the target communication link for message transmission from multiple communication links based on the message type of the response message. Message types can include, for example, command types and non-command types. Command type response messages can include one or more commands to be executed locally by the client device 110, such as commands to turn Bluetooth on or off, volume adjustment commands, etc. After these commands are sent to the client device 110, the client device 110 can receive and execute these one or more commands. Non-command type response messages can include content to be presented to the user, such as text or voice. Non-command type response messages may be presented in the interactive interface and / or played to the user (e.g., in the case of voice).

[0084] Background service 202 may, for example, determine the MQTT link as the target communication link in response to a message type of the reply message being an instruction. For instance, if the reply message is an instruction to be executed by client device 110, background service 202 may determine to send the instruction to client device 110 via the MQTT link. Background service 202 may also, for instance, determine the target communication link from UDP and HTTP links in response to a message type of the reply message being a non-instruction type (e.g., determining the target communication link from UDP and HTTP links based on the networking capabilities of client device 110). For instance, if the reply message is a regular reply text or voice, background service 202 may determine to send the reply text or voice to client device 110 via a UDP link or an HTTP link.

[0085] The background service 202 can send the response message to the client device 110 via a determined target communication link. In some embodiments, if the response message is determined to be a non-command type and the target communication link is determined to be a first communication link (e.g., a UDP link), the background service 202 can collectively refer to the response voice or response text determined by the machine learning model as the response determined by the background service using the machine learning model (specifically, a language model), and generate at least one response message based on the response according to a predetermined data structure corresponding to the UDP protocol. It is understood that each response message in this case includes at least a part of the response. It is also understood that the responses included in different response messages do not overlap. As an example, the background service 202 can generate 5 response messages based on the response, that is, each response message includes 1 / 5 of the response.

[0086] The backend service 202 can, for example, determine the amount of content included in each response message according to a predetermined length. As an example, if the response is audio, the predetermined length can be a predetermined duration, and each response message includes the predetermined duration of audio from the audio response. If the response is text, the predetermined length can be a predetermined number of texts, and each response message includes the predetermined number of texts from the response.

[0087] Each response message may also include an identifier of the session that received the query request. The session identifier may include any suitable identifier such as text, symbols, images, icons, etc., and may also be referred to as a session ID. Each response message may also include the content type of the response, which includes at least voice and / or text types. As an example, the content type of the response may also include any suitable type such as image or video. Each response message may also include the order of the response message within at least one response message. Each response message may also include whether the response message is the last request message in at least one request message. Each response message may also include the length of at least a portion of the response included in the response message. The length of at least a portion of the response included in the response message is also the predetermined length mentioned above. For example, if each response message includes a predetermined duration of audio in the response voice, the length of at least a portion of the response included in the response message is also the predetermined duration. It is understood that each response message may also include one or more of the above-mentioned session identifier, response content type, order, whether it is the last request message, and length, and this disclosure does not limit this. Referring to Table 2, Table 2 shows an example of a response message generated according to a predetermined data structure:

[0088] Table 2

[0089]

[0090] If the target communication link for message sending is determined to include a UDP link, in scheme 206, the background service 202 can send (223) the response message to the server device 120 via the UDP link.

[0091] In some embodiments, if it is determined that the response message is a non-command type and the target communication link is determined to be an HTTP link (i.e., it is determined that the response message will be sent via an HTTP link, in which case the response message can be directly response text or response voice), the background service 202 can provide the client device 110 with a first message notification including the access link of the response message via an MQTT link, and send the response message to the client device 110 via the HTTP link based on the access link. This corresponds to Figure 2 Scheme 207 is shown.

[0092] Specifically, the background service 202 can save the response message in a predetermined format (i.e., save the response text or response voice generated by the machine learning model in a predetermined format), which is a format supported by the client device 110. For example, if the response message is response voice, the predetermined format can be MP3 format; if the response message is response text, the predetermined format can be TXT format, etc. The predetermined format can be pre-set. The background service 202 can then generate (224) an access link for the saved response message and provide (225) the access link to the MQTT service 201.

[0093] The MQTT service 201 in server device 120 can, in response to obtaining an access link, determine a first message notification including the access link based on the access link. Client device 110 can receive (226) the first message notification provided by server device 120 via an MQTT link. The first message notification includes an access link to a response message, and the response message is of a non-command type. For example, client device 110 can, in response to receiving the first message notification, determine that it wants to obtain the response message via an HTTP link (i.e., determine that the target communication link for data reception is an HTTP link). If the first message notification is received and it is determined that the first message notification includes an access link, client device 110 can, via the HTTP link, receive (227) the response message from server device 120 based on the access link. For example, client device 110 can, in response to receiving the first message notification, send an information retrieval request to server device 120, and then receive the response message from server device 120 via the HTTP link. Alternatively, client device 110 can also verbally play the response message via a speaker.

[0094] In addition to the client device 110 actively obtaining a response message based on the access link, in some embodiments, the client device 110 may provide the access link to the user first after receiving the first message notification. For example, the client device 110 may provide the access link in the user interface. Alternatively, the client device 110 may also play the access link aloud via a speaker. In response to receiving a user's trigger for the access link, the client device 110 may receive a response message from the server device 120 via an HTTP link based on the access link.

[0095] Client device 110 can provide a response message (228) to the user. The provided response message can be, for example, a reply from the digital assistant to a query request in a session. As an example, in a session between the user and the digital assistant, the query request can be presented as a session message from the user, and the response message can be presented as a session message from the digital assistant.

[0096] In some embodiments, if the response message is directly response text or response voice, the client device 110 can directly provide the response message to the user. For example, the response text can be presented directly via the user interface, or the response voice can be played via a speaker. In some embodiments, if the response message received by the client device 110 is a response message received via a UDP link (the response message is a response message generated based on response text / response voice, and each response message includes at least a portion of the response text / response voice), the client device 110 can, in response to receiving the response message, parse the response message according to a predetermined data structure corresponding to the UDP protocol to determine the response text / response voice to be provided.

[0097] The above description refers to data transmission of non-command type response messages. The following description continues with data transmission of command type response messages. As mentioned earlier, background service 202 can determine the MQTT link as the target communication link in response to a command type response message. In this case, the sending of the response message will correspond to... Figure 2 Scheme 208. The background service 202 can obtain (229) instruction-type response messages from the language model 204. Instruction-type response messages can be regarded as instructions. In some embodiments, the background service 202 can also obtain non-instruction-type response messages (e.g., response text) from the language model 204 and determine whether the response message will trigger an instruction. In response to the fact that the response message can trigger an instruction, the background service 202 can determine the corresponding instruction based on the response message. The instruction indicated by the instruction-type response message and the instruction triggered by the non-instruction-type response message can be collectively referred to as the instruction associated with the response message.

[0098] The background service 202 can then provide (230) the instruction associated with the response message to the MQTT service 201. The MQTT service 201 in the server device 120 can, in response to receiving the instruction, determine a second message notification including the instruction based on the instruction. The client device 110 can receive (231) the second message notification provided by the server device 120 via the MQTT link. The client device 110 can obtain the instruction based on the second message notification and execute (232) the instruction.

[0099] In summary, the client and server devices can transmit query requests via a predetermined, high-speed first communication link, and transmit response messages to the query requests via a second communication link determined from multiple communication links. The first and second communication links can be the same or different links. That is, query requests and response messages can be transmitted via different communication links. Therefore, the client and server devices can select an appropriate communication link from multiple links to transmit response messages, which can improve the quality and efficiency of data transmission during the response process, thereby improving the quality and efficiency of response processing.

[0100] Figure 3 A flowchart of a method 300 for response processing according to some embodiments of the present disclosure is shown. Method 300 may be implemented at client device 110.

[0101] In box 310, in response to receiving a query request for the digital assistant in a session with the digital assistant, the client device 110 sends the query request to the server device via a predetermined first communication link.

[0102] In box 320, client device 110 receives a response message for a query request from server device via a target communication link among a plurality of communication links, the plurality of communication links including at least a first communication link.

[0103] In box 330, the client device 110 provides a response message as a digital assistant's reply to the query request.

[0104] In some embodiments, the target communication link is determined from a plurality of communication links based on at least one of the following: the networking capability of the client device, the link indication sent by the client device to the server device, and the message notification from the server device.

[0105] In some embodiments, method 300 further includes: in response to receiving a selection of a target communication link among a plurality of communication links, providing a link indication to a server device, the link indication including an indication of the selected target communication link; or in response to determining that the networking capability of a client device is higher than a first capability level, sending a link indication to the server device providing an indication of a first communication link; or in response to determining that the networking capability of a client device is lower than a second capability level, providing a link indication to the server device indicating a second communication link, the communication reliability of the second communication link being higher than the communication reliability of the first communication link.

[0106] In some embodiments, a link indication is provided to the server device via Message Queuing Telemetry (MQTT) link transmission. In some embodiments, the target communication link includes the communication link indicated by the link indication.

[0107] In some embodiments, the target communication link includes a communication link corresponding to the Hypertext Transfer Protocol (HTTP) that is different from the first communication link, and receiving a response message from the server device via the target communication link includes: receiving a first message notification provided by the server device via an MQTT link, the first message notification including an access link for the response message, and the response message being a non-command type; and in response to receiving the first message notification, receiving the response message from the server device via the communication link corresponding to the HTTP protocol based on the access link.

[0108] In some embodiments, the target communication link includes an MQTT link different from the first communication link, and receiving a response message from the server device via the target communication link includes receiving a second message notification provided by the server device via the MQTT link, the second message notification including a response message of the instruction type.

[0109] In some embodiments, the first communication link includes a communication link based on the User Datagram Protocol (UDP), and sending a query request to a server device via the predetermined first communication link includes: generating at least one request message based on the query request according to a predetermined data structure corresponding to the UDP protocol, each request message including at least a portion of the query request and further including at least one of the following: a session identifier, a content type of the query request, the content type including at least a voice type and / or a text type, the order of the request message in the at least one request message, whether the request message is the last request message in the at least one request message; and sending the at least one request message to the server device via the first communication link.

[0110] Figure 4 A flowchart of a method 400 for response processing according to other embodiments of the present disclosure is shown. Method 400 may be implemented at server device 120.

[0111] In box 410, server device 120 receives a query request for the digital assistant from client device via a predetermined first communication link.

[0112] In box 420, server device 120 determines the response message corresponding to the query request.

[0113] In box 430, server device 120 sends a response message to client device via a target communication link among multiple communication links, the multiple communication links including at least a first communication link.

[0114] In some embodiments, method 400 further includes determining a target communication link from a plurality of communication links based on at least one of the following: the networking capability of the client device, the link indication sent by the client device to the server device, and the message type of the response message.

[0115] In some embodiments, determining a target communication link for message transmission from multiple communication links based on the networking capabilities of a client device includes: determining the networking capabilities of the client device; determining a second communication link as the target communication link in response to the client device's networking capabilities being lower than a third capability level, wherein the communication reliability of the second communication link is higher than that of the first communication link; and determining a first communication link as the target communication link in response to the client device's networking capabilities being higher than a fourth capability level.

[0116] In some embodiments, receiving a query request includes receiving a plurality of request messages corresponding to the query request, each request message including at least a portion of the query request; and determining the network capability of the client device includes: in response to receiving the plurality of request messages corresponding to the query request, determining the network capability of the client device based on the order indicated by the plurality of request messages and the receiving order of the plurality of request messages.

[0117] In some embodiments, determining the target communication link for message transmission from a plurality of communication links based on a link indication sent by the client device to the server device includes: in response to receiving the link indication, determining the communication link indicated by the link indication as the target communication link.

[0118] In some embodiments, the plurality of communication links further include a Message Queuing Telemetry Transport (MQTT) link, and wherein determining the target communication link for message sending from the plurality of communication links based on the message type of the acknowledgment message includes: determining the MQTT link as the target communication link in response to the acknowledgment message being of the instruction type.

[0119] In some embodiments, the target communication link includes an MQTT link different from the first communication link, and the method 400 further includes: in response to determining that the response message is a non-instruction type and determining that the response message is sent via the communication link corresponding to the HTTP protocol, providing a first message notification to the client device via the MQTT link, the first message notification including an access link for the response message; and sending the response message to the client device via the communication link corresponding to the HTTP protocol based on the access link.

[0120] In some embodiments, the first communication link includes a communication link based on the User Datagram Protocol (UDP), and determining the response message corresponding to the query request includes: parsing multiple request messages according to a predetermined data structure corresponding to the UDP protocol to determine the query request; and using a trained language model to determine the response message corresponding to the query request.

[0121] In some embodiments, if the target communication link is a first communication link, determining the response message corresponding to the query request based on the query request includes: using a trained language model to determine the response corresponding to the query request based on the query request; and generating at least one response message based on the response according to a predetermined data structure corresponding to the UDP protocol, each response message including at least a portion of the response and further including at least one of the following: an identifier of the session that received the query request, the content type of the response, the content type including at least voice type and text type, the order of the response message in the at least one response message, whether the response message is the last response message in the at least one response message, and the length of the at least portion of the response included in the response message.

[0122] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 5 An exemplary structural block diagram of an apparatus 500 for response processing according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a client device 110. Various modules / components in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0123] like Figure 5 As shown, the device 500 includes a query request sending module 510, configured to send the query request to a server device via a predetermined first communication link in response to receiving a query request for the digital assistant during a session with the digital assistant. The device 500 also includes a response message receiving module 520, configured to receive a response message for the query request from the server device via a target communication link among a plurality of communication links, at least including the first communication link. The device 500 further includes a response message providing module 530, configured to provide the response message as a reply from the digital assistant to the query request.

[0124] In some embodiments, the target communication link is determined from a plurality of communication links based on at least one of the following: the networking capability of the client device, the link indication sent by the client device to the server device, and the message notification from the server device.

[0125] In some embodiments, the apparatus 500 further includes: a first link indication sending module configured to provide a link indication to a server device in response to receiving a selection of a target communication link among a plurality of communication links, the link indication including an indication of the selected target communication link; or a second link indication sending module configured to send a link indication providing an indication of a first communication link to a server device in response to determining that the networking capability of a client device is higher than a first capability level; or a third link indication sending module configured to provide a link indication of a second communication link to a server device in response to determining that the networking capability of a client device is lower than a second capability level, the communication reliability of the second communication link being higher than the communication reliability of the first communication link.

[0126] In some embodiments, a link indication is provided to the server device via Message Queuing Telemetry (MQTT) link transmission. In some embodiments, the target communication link includes the communication link indicated by the link indication.

[0127] In some embodiments, the target communication link includes a communication link corresponding to the Hypertext Transfer Protocol (HTTP) that is different from the first communication link, and the response message receiving module 520 is further configured to: receive a first message notification provided by the server device via an MQTT link, the first message notification including an access link for a response message, and the response message being a non-command type; and in response to receiving the first message notification, receive a response message from the server device based on the access link via the communication link corresponding to the HTTP protocol.

[0128] In some embodiments, the target communication link includes an MQTT link different from the first communication link, and the response message receiving module 520 is further configured to receive a second message notification provided by the server device via the MQTT link, the second message notification including a response message of the instruction type.

[0129] In some embodiments, the first communication link includes a communication link based on the User Datagram Protocol (UDP), and the query request sending module 510 is further configured to: generate at least one request message based on a query request according to a predetermined data structure corresponding to the UDP protocol, each request message including at least a part of the query request and including at least one of the following: a session identifier, a content type of the query request, the content type including at least voice type and text type, the order of the request message in the at least one request message, whether the request message is the last request message in the at least one request message; and send the at least one request message to the server device via the first communication link.

[0130] Figure 6 An exemplary structural block diagram of an apparatus 600 for response processing according to other embodiments of the present disclosure is shown. The apparatus 600 may be implemented as or included in the server device 120. The various modules / components in the apparatus 600 may be implemented by hardware, software, firmware, or any combination thereof.

[0131] like Figure 6 As shown, device 600 includes a query request receiving module 610, configured to receive a query request for a digital assistant from a client device via a predetermined first communication link. Device 600 also includes a response message determining module 620, configured to determine a response message corresponding to the query request. Device 600 further includes a response message sending module 630, configured to send the response message to the client device via a target communication link among a plurality of communication links, the plurality of communication links including at least the first communication link.

[0132] In some embodiments, the target communication link is determined from a plurality of communication links based on at least one of the following: the networking capability of the client device, the link indication sent by the client device to the server device, and the message type of the response message.

[0133] In some embodiments, determining a target communication link for message transmission from multiple communication links based on the networking capabilities of a client device includes: determining the networking capabilities of the client device; determining a second communication link as the target communication link in response to the client device's networking capabilities being lower than a third capability level, wherein the communication reliability of the second communication link is higher than that of the first communication link; and determining a first communication link as the target communication link in response to the client device's networking capabilities being higher than a fourth capability level.

[0134] In some embodiments, receiving a query request includes receiving a plurality of request messages corresponding to the query request, each request message including at least a part of the query request; and determining the network capability of the client device includes: in response to receiving the plurality of request messages corresponding to the query request, determining the network capability of the client device based on the order indicated by the plurality of request messages and the receiving order of the plurality of request messages.

[0135] In some embodiments, determining the target communication link for message transmission from a plurality of communication links based on a link indication sent by the client device to the server device includes: in response to receiving the link indication, determining the communication link indicated by the link indication as the target communication link.

[0136] In some embodiments, the plurality of communication links further include a Message Queuing Telemetry Transport (MQTT) link, and determining the target communication link for message sending from the plurality of communication links based on the message type of the acknowledgment message includes: in response to the acknowledgment message being of the instruction type, determining the MQTT link as the target communication link.

[0137] In some embodiments, the target communication link includes an MQTT link different from the first communication link, and the apparatus 600 further includes: a message notification providing module configured to provide a first message notification to the client device via the MQTT link in response to determining that the response message is a non-instruction type and determining that the response message will be sent via the communication link corresponding to the HTTP protocol, the first message notification including an access link for the response message; and a message sending module configured to send the response message to the client device via the communication link corresponding to the HTTP protocol based on the access link.

[0138] In some embodiments, the first communication link includes a communication link based on the User Datagram Protocol (UDP), and the response message determination module 620 is further configured to: parse multiple request messages according to a predetermined data structure corresponding to the UDP protocol to determine a query request; and use a trained language model to determine a response message corresponding to the query request based on the query request.

[0139] In some embodiments, if the target communication link is a first communication link, the response message determination module 620 is further configured to: determine a response corresponding to the query request based on the query request using a trained language model; and generate at least one response message based on the response according to a predetermined data structure corresponding to the UDP protocol, each response message including at least a portion of the response and including at least one of the following: an identifier of the session that received the query request, the content type of the response, the content type including at least voice type and text type, the order of the response message in the at least one response message, whether the response message is the last response message in the at least one response message, and the length of at least a portion of the response included in the response message.

[0140] The units and / or modules included in device 500 / 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units and / or modules in device 500 / 600 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0141] It should be understood that one or more steps in the above methods can be performed by suitable electronic devices or combinations of electronic devices. Such electronic devices or combinations of electronic devices may include, for example, […]. Figure 1 The client device 110 and / or server device 120 in the middle.

[0142] Figure 7 A block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 7 The electronic device 700 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 7 The electronic device 700 shown can be used to achieve Figure 1 The client device 110 and / or the server device 120.

[0143] like Figure 7As shown, electronic device 700 is in the form of a general-purpose electronic device. Components of electronic device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 700.

[0144] Electronic device 700 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 720 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 730 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 700.

[0145] Electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 7 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 720 may include computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0146] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 700 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 700 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0147] Input device 750 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 760 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 700 can also communicate with one or more external devices (not shown) via communication unit 740 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 700, or with any device that enables electronic device 700 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0148] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0149] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0150] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0151] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some, as newer, implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0153] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for response processing, implemented at a client device, comprising: In response to receiving a query request for the digital assistant during a session with the digital assistant, the query request is sent to the server device via a predetermined first communication link; The server receives a response message for the query request from the server device via a target communication link among multiple communication links, wherein the multiple communication links include at least the first communication link; as well as The response message is provided as the digital assistant's reply to the query request.

2. The method of claim 1, wherein the target communication link is determined from the plurality of communication links based on at least one of the following: The network connectivity of the client device The link indication sent by the client device to the server device Message notification from the server device.

3. The method according to claim 2, further comprising: In response to receiving a selection of a target communication link among the plurality of communication links, the server device is provided with a link indication, the link indication including an indication of the selected target communication link; or In response to determining that the client device's networking capability is higher than a first capability level, a link indication providing the first communication link is sent to the server device; or In response to determining that the network capability of the client device is lower than the second capability level, the server device is provided with a link indication indicating a second communication link, wherein the communication reliability of the second communication link is higher than that of the first communication link.

4. The method of claim 3, wherein the link indication is provided to the server device via Message Queue Telemetry Transmission (MQTT) link.

5. The method of claim 3, wherein the target communication link includes the communication link indicated by the link indicator.

6. The method of claim 1, wherein the target communication link includes a communication link corresponding to the Hypertext Transfer Protocol (HTTP) different from the first communication link, and wherein receiving a response message from the server device via the target communication link includes: The system receives a first message notification provided by the server device via an MQTT link. The first message notification includes an access link to the response message, and the response message is a non-command type. as well as In response to receiving the first message notification, the response message is received from the server device via the communication link corresponding to the HTTP protocol, based on the access link.

7. The method of claim 1, wherein the target communication link includes an MQTT link different from the first communication link, and wherein receiving a response message from the server device via the target communication link includes: The second message notification, which includes the instruction-type response message, is received via the MQTT link from the server device.

8. The method of claim 1, wherein the first communication link includes a communication link based on the User Datagram Protocol (UDP), and wherein sending a query request to a server device via the predetermined first communication link comprises: Based on the query request, at least one request message is generated according to the predetermined data structure corresponding to the UDP protocol. Each request message includes at least a portion of the query request and also includes at least one of the following: The identifier of the session, The content type of the query request includes at least voice and text types. The order of the request messages in the at least one request message. Whether the request message is the last request message among the at least one request messages; as well as The at least one request message is sent to the server device via the first communication link.

9. A response provision method, implemented at a server-side device, comprising: Receive query requests for the digital assistant from the client device via a predetermined first communication link; Determine the response message corresponding to the query request; as well as The response message is sent to the client device via a target communication link among multiple communication links, wherein the multiple communication links include at least the first communication link.

10. The method of claim 9, further comprising determining the target communication link from the plurality of communication links based on at least one of the following: The network connectivity of the client device The link indication sent by the client device to the server device The message type of the response message.

11. The method of claim 10, wherein determining the target communication link for message transmission from the plurality of communication links based on the networking capability of the client device comprises: Determine the network connectivity of the client device; In response to the fact that the network capability of the client device is lower than the third capability level, the second communication link is determined as the target communication link, and the communication reliability of the second communication link is higher than that of the first communication link; as well as In response to the client device's networking capability being higher than the fourth capability level, the first communication link is identified as the target communication link.

12. The method of claim 11, wherein receiving the query request includes receiving a plurality of request messages corresponding to the query request, each request message including at least a portion of the query request; and Determining the network connectivity of the client device includes: In response to receiving multiple request messages corresponding to the query request, the network capability of the client device is determined based on the sorting indicated by the multiple request messages and the receiving order of the multiple request messages.

13. The method of claim 10, wherein determining the target communication link for message transmission from the plurality of communication links based on a link indication sent by the client device to the server device comprises: In response to receiving the link indication, the communication link indicated by the link indication is determined as the target communication link.

14. The method of claim 10, wherein the plurality of communication links further comprises a Message Queuing Telemetry Transport (MQTT) link, and wherein determining the target communication link for message sending from the plurality of communication links based on the message type of the acknowledgment message comprises: If the message type of the response message is an instruction type, the MQTT link is determined as the target communication link.

15. The method of claim 9, wherein the target communication link includes an MQTT link different from the first communication link, and the method further includes: In response to determining that the response message is a non-command type and determining that the response message is sent via the communication link corresponding to the HTTP protocol, a first message notification is provided to the client device via the MQTT link, the first message notification including the access link of the response message; as well as The response message is sent to the client device via the communication link corresponding to the HTTP protocol, based on the access link.

16. An apparatus for response processing, implemented at a client device, comprising: The query request sending module is configured to send the query request to the server device via a predetermined first communication link in response to receiving a query request for the digital assistant in a session with the digital assistant. The response message receiving module is configured to receive a response message for the query request from the server device via a target communication link among a plurality of communication links, wherein the plurality of communication links includes at least the first communication link; as well as A response message providing module is configured to provide the response message as the digital assistant's reply to the query request.

17. An apparatus for response processing, implemented at a server device, comprising: The query request receiving module is configured to receive query requests for the digital assistant from the client device via a predetermined first communication link; The response message determination module is configured to determine the response message corresponding to the query request; as well as The response message sending module is configured to send the response message to the client device via a target communication link among a plurality of communication links, wherein the plurality of communication links includes at least the first communication link.

18. An electronic device comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8, and / or, the method according to any one of claims 9 to 15, when executed by the at least one processing unit.

19. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8, and / or, the method according to any one of claims 9 to 15.

20. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 8, and / or, the method according to any one of claims 9 to 15.

Images