Voice response method, voice response system, client and server

Abstract

The invention provides a voice response method, a voice response system, a client and a server, the method is applied to the voice response system, the voice response system comprises the client and the server, and the client comprises a sound receiver, a broadcaster, a display screen and a terminal processing unit. The method comprises the steps that a terminal processing unit obtains text data corresponding to voice data according to the voice data collected by a sound receiver and sends the text data to a server side; the server side classifies the text data to obtain a classification result, and the classification result indicates attributes of the text data; the server side executes processing logic corresponding to the classification result on the text data according to the classification result to obtain a processing result corresponding to the attribute of the text data, and sends the processing result to the terminal processing unit; and the terminal processing unit controls the broadcaster to play the audio and / or the display screen to play the display content according to the processing result. According to the method, the response speed of the voice response system can be increased on the premise of maintaining the quality of the voice response method.

Description

Technical Field

[0001] This application belongs to the field of voice response technology, specifically relating to a voice response method, a voice response system, a client, and a server. Background Technology

[0002] With the development of technology, service robots have emerged to serve users. These robots can be applied in various fields, such as hotels and airports, providing users with services including inquiries through voice response systems. Currently, voice response systems can understand user inquiries or commands using language models and provide real-time responses. However, because high-quality language models have a long response time to user inquiries or commands, the real-time response speed of voice response systems to customers is slow, thus affecting the user experience.

[0003] Therefore, improving the real-time response speed of voice response systems is an urgent problem to be solved. Summary of the Invention

[0004] To address the problems existing in the prior art, a voice response method, a voice response system, a client, and a server are proposed. By using this voice response method, voice response system, client, and server, the response speed of the voice response system to customers can be improved.

[0005] This application provides the following solutions.

[0006] In a first aspect, this application provides a voice response method applied to a voice response system, the voice response system including a client and a server, the client including a microphone, a speaker, a display screen, and a terminal processing unit, the method including:

[0007] The terminal processing unit obtains the text data corresponding to the voice data collected by the microphone and sends the text data to the server.

[0008] The server classifies the text data and obtains the classification results, which indicate the attributes of the text data.

[0009] Based on the classification results, the server executes the processing logic corresponding to the classification results on the text data, obtains the processing results corresponding to the attributes of the text data, and sends the processing results to the terminal processing unit.

[0010] Based on the processing results, the terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content.

[0011] In some possible embodiments, the processing result includes a response instruction. The server executes processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0012] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit.

[0013] In some possible embodiments, the processing result includes response text data. The server performs processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0014] If the classification result indicates that the text data is a question attribute, the server will input the text data into the language model to obtain the response text data.

[0015] In some possible embodiments, the server side includes a text processing module;

[0016] The server classifies the text data and obtains the classification results, including:

[0017] The text processing module classifies the text data and obtains the classification results;

[0018] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit, including:

[0019] If the classification result indicates that the text data is an instruction attribute, the text processing module will send the instruction corresponding to the text data to the terminal processing unit.

[0020] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0021] If the classification result indicates that the text data is a question attribute, the server inputs the text data into the language model to obtain the response text data, including:

[0022] If the classification result indicates that the text data is a question attribute, the text processing module sends the text data to the language response module;

[0023] The language response module inputs text data into the language model, obtains response text data, and sends the response text data to the terminal processing unit through the text processing module.

[0024] In some possible embodiments, it also includes:

[0025] The language response module uses a language model to perform semantic understanding of text data and obtain the understanding results.

[0026] If the interpretation result indicates that the text data is an instruction attribute, the language response module will send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0027] If the interpretation result indicates that the text data is a question attribute, the language response module generates the corresponding response text data through the language model, and sends the response text data to the terminal processing unit through the text processing module.

[0028] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0029] The language module includes one or more sub-language models, each of which corresponds to one or more language categories. The language response module inputs text data into the language model to obtain response text data, including:

[0030] The language response module inputs text data into one or more sub-language models corresponding to the language identifier, and obtains the response text data corresponding to the language identifier.

[0031] In some possible embodiments, the terminal processing unit controls the speaker to play audio and / or the display screen to display content based on the reply text data, including:

[0032] The terminal processing unit obtains the corresponding voice data based on the response text data.

[0033] The terminal processing unit controls the broadcaster to play the response voice data.

[0034] In some possible embodiments, the terminal processing unit obtains text data corresponding to the voice data collected by the microphone, including:

[0035] The terminal processing unit decodes the voice data to obtain the corresponding text data.

[0036] Alternatively, the terminal processing unit uploads the voice data to the backend server for decoding, and obtains the text data returned by the backend server.

[0037] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit controls the display screen to play displayed content based on the reply text data, including:

[0038] The terminal processing unit controls the display screen to show the text corresponding to the reply text data based on the reply text data, and controls the interactive objects displayed on the display screen to interact with the user based on the reply text data.

[0039] In some possible embodiments, the client may also include a camera;

[0040] The camera is used to collect user image data;

[0041] The terminal processing unit obtains the corresponding text data based on the voice data collected by the microphone, including:

[0042] The terminal processing unit extracts key voice data from the microphone based on the user's image data.

[0043] The terminal processing unit obtains the text data corresponding to the key voice data based on the key voice data.

[0044] Secondly, this application provides a voice response method, which is applied to a client and communicates with a server. The client includes a microphone, a speaker, a display screen, and a terminal processing unit. The method includes:

[0045] The terminal processing unit obtains the text data corresponding to the voice data based on the voice data collected by the microphone;

[0046] The terminal processing unit sends text data to the server so that the server can classify the text data and obtain the classification result, which indicates the attributes of the text data.

[0047] The classification results are used to assist the server in performing the processing logic corresponding to the classification results on the text data, and to obtain the processing results corresponding to the attributes of the text data.

[0048] Based on the processing results, the terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content.

[0049] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0050] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0051] In some possible embodiments, the server side includes a text processing module;

[0052] The text processing module is configured to classify text data and obtain classification results;

[0053] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0054] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0055] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module;

[0056] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0057] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0058] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0059] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0060] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0061] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0062] In some possible embodiments, the terminal processing unit controls the speaker to play audio and / or the display screen to display content based on the reply text data, including:

[0063] The terminal processing unit obtains the corresponding voice data based on the response text data.

[0064] The terminal processing unit controls the broadcaster to play the response voice data.

[0065] In some possible embodiments, the terminal processing unit obtains text data corresponding to the voice data collected by the microphone, including:

[0066] The terminal processing unit decodes the voice data to obtain the corresponding text data.

[0067] Alternatively, the terminal processing unit uploads the voice data to the backend server for decoding, and obtains the text data returned by the backend server.

[0068] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit controls the display screen to play displayed content based on the reply text data, including:

[0069] The terminal processing unit controls the display screen to show the text corresponding to the reply text data based on the reply text data, and controls the interactive objects displayed on the display screen to interact with the user based on the reply text data.

[0070] In some possible embodiments, the client may also include a camera;

[0071] The camera is used to collect user image data;

[0072] The terminal processing unit obtains the corresponding text data based on the voice data collected by the microphone, including:

[0073] The terminal processing unit extracts key voice data from the microphone based on the user's image data.

[0074] The terminal processing unit obtains the text data corresponding to the key voice data based on the key voice data.

[0075] Thirdly, this application provides a voice response method, which is applied on a server side, the server side communicating with a client side, the client side including a microphone, a broadcasting device, a display screen and a terminal processing unit, the method including:

[0076] The server receives text data sent by the terminal processing unit, and the text data is generated based on the voice data collected by the microphone.

[0077] The server classifies the text data and obtains the classification results, which indicate the attributes of the text data.

[0078] Based on the classification results, the server executes the processing logic corresponding to the classification results on the text data to obtain the processing results corresponding to the attributes of the text data;

[0079] The server sends the processing results to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play content based on the processing results.

[0080] In some possible embodiments, the processing result includes a response instruction. The server executes processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0081] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit.

[0082] In some possible embodiments, the processing result includes response text data. The server performs processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0083] If the classification result indicates that the text data is a question attribute, the server will input the text data into the language model to obtain the response text data.

[0084] In some possible embodiments, the server side includes a text processing module;

[0085] The server classifies the text data and obtains the classification results, including:

[0086] The text processing module classifies the text data and obtains the classification results;

[0087] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit, including:

[0088] If the classification result indicates that the text data is an instruction attribute, the text processing module will send the instruction corresponding to the text data to the terminal processing unit.

[0089] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0090] If the classification result indicates that the text data is a question attribute, the server inputs the text data into the language model to obtain the response text data, including:

[0091] If the classification result indicates that the text data is a question attribute, the text processing module sends the text data to the language response module;

[0092] The language response module inputs text data into the language model, obtains response text data, and sends the response text data to the terminal processing unit through the text processing module.

[0093] In some possible embodiments, it also includes:

[0094] The language response module uses a language model to perform semantic understanding of text data and obtain the understanding results.

[0095] If the interpretation result indicates that the text data is an instruction attribute, the language response module will send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0096] If the interpretation result indicates that the text data is a question attribute, the language response module generates the corresponding response text data through the language model, and sends the response text data to the terminal processing unit through the text processing module.

[0097] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0098] The language module includes one or more sub-language models, each of which corresponds to one or more language categories. The language response module inputs text data into the language model to obtain response text data, including:

[0099] The language response module inputs text data into one or more sub-language models corresponding to the language identifier, and obtains the response text data corresponding to the language identifier.

[0100] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0101] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0102] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0103] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0104] In some possible embodiments, the client may also include a camera;

[0105] The camera is used to collect user image data;

[0106] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone, including:

[0107] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0108] Fourthly, this application provides a voice response system, which includes a client and a server. The client includes a microphone, a speaker, a display screen, and a terminal processing unit.

[0109] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone and send the text data to the server.

[0110] The server is configured to classify text data, obtain classification results, and the classification results indicate the attributes of the text data.

[0111] The server is configured to perform processing logic corresponding to the classification results on the text data based on the classification results, obtain the processing results corresponding to the attributes of the text data, and send the processing results to the terminal processing unit.

[0112] The terminal processing unit is configured to control the speaker to play audio and / or the display screen to play displayed content based on the processing results.

[0113] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0114] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0115] In some possible embodiments, the server side includes a text processing module;

[0116] The text processing module is configured to classify text data and obtain classification results;

[0117] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0118] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0119] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module.

[0120] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0121] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0122] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0123] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0124] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0125] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0126] In some possible embodiments, the terminal processing unit is configured to obtain the reply voice data corresponding to the reply text data based on the reply text data;

[0127] The terminal processing unit is configured to control the broadcaster to play back the response voice data.

[0128] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0129] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0130] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0131] In some possible embodiments, the client may also include a camera;

[0132] The camera is used to collect user image data;

[0133] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0134] Fifthly, this application provides a client that communicates with a server. The client includes a microphone, a loudspeaker, a display screen, and a terminal processing unit.

[0135] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone;

[0136] The terminal processing unit is configured to send text data to the server so that the server can classify the text data and obtain the classification result, which indicates the attributes of the text data.

[0137] The classification results are configured to assist the server in performing the processing logic corresponding to the classification results on the text data, and to obtain the processing results corresponding to the attributes of the text data.

[0138] The terminal processing unit is configured to control the speaker to play audio and / or the display screen to play displayed content based on the processing results.

[0139] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0140] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0141] In some possible embodiments, the server side includes a text processing module;

[0142] The text processing module is configured to classify text data and obtain classification results;

[0143] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0144] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0145] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module;

[0146] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0147] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0148] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0149] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0150] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0151] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0152] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0153] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0154] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0155] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0156] In some possible embodiments, the client may also include a camera;

[0157] The camera is used to collect user image data;

[0158] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0159] Sixthly, this application provides a server-side component that communicates with a client-side component. The client-side component includes a microphone, a loudspeaker, a display screen, and a terminal processing unit.

[0160] The server is configured to receive text data sent by the terminal processing unit, and the text data is generated based on the voice data collected by the microphone.

[0161] The server is configured to classify text data, obtain classification results, and the classification results indicate the attributes of the text data.

[0162] The server is configured to perform processing logic corresponding to the classification results on the text data based on the classification results, and obtain the processing results corresponding to the attributes of the text data.

[0163] The server is configured to send processing results to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play content based on the processing results.

[0164] In some possible embodiments, the processing result includes a response command;

[0165] If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0166] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0167] In some possible embodiments, the server side includes a text processing module;

[0168] The server is configured to classify text data and obtain classification results, including:

[0169] The text processing module is configured to classify text data and obtain classification results;

[0170] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0171] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0172] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module.

[0173] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0174] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0175] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0176] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0177] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0178] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0179] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0180] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0181] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0182] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0183] In some possible embodiments, the client may also include a camera;

[0184] The camera is used to collect user image data;

[0185] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone, including:

[0186] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0187] The voice response method provided in this application converts voice data into text data, and then the server classifies the attributes of the text data to obtain classification results. The server can classify text with different attributes based on these attributes to obtain processing results. By classifying the text data before processing, the server can better identify and process the text data in a targeted manner, thereby accelerating the response speed of the voice response system to customers while maintaining the quality of the voice response method. Other advantages of this application will be explained in more detail below with reference to the accompanying drawings.

[0188] It should be understood that the above description is merely an overview of the technical solution of this application, so as to enable a clearer understanding of the technical means of this application and thus allow for its implementation in accordance with the contents of the specification. To make the above and other objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application are illustrated below. Attached Figure Description

[0189] By reading the detailed description of the exemplary embodiments below, those skilled in the art will understand the advantages and benefits described herein, as well as other advantages and benefits. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0190] Figure 1 A framework diagram of a voice response system provided in an embodiment of this application;

[0191] Figure 2 A flowchart of a voice response method provided in an embodiment of this application;

[0192] Figure 3 A flowchart illustrating another voice response method provided in this application embodiment;

[0193] Figure 4 A schematic diagram of a voice response method provided in an embodiment of this application;

[0194] Figure 5 A signaling diagram of a voice response method provided in an embodiment of this application;

[0195] Figure 6 A flowchart of a voice response method provided in an embodiment of this application;

[0196] Figure 7 A flowchart illustrating another voice response method provided in this application embodiment;

[0197] Figure 8 A schematic diagram of a voice response system provided in an embodiment of this application;

[0198] Figure 9 A schematic diagram of a client provided in an embodiment of this application;

[0199] Figure 10 This is a schematic diagram of a server-side embodiment provided in this application.

[0200] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0201] Exemplary embodiments of this application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0202] In the description of embodiments of this application, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of the disclosed features, figures, steps, behaviors, components, portions or combinations thereof in this specification, and do not exclude the possibility of the presence of one or more other features, figures, steps, behaviors, components, portions or combinations thereof.

[0203] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B at the same time, and B alone.

[0204] The terms "first," "second," etc., are used only for ease of description to distinguish identical or similar technical features and should not be construed as indicating or implying the relative importance or number of these technical features. Therefore, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this application, unless otherwise stated, the term "multiple" means two or more.

[0205] It should also be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0206] The method provided in this application is applied to a voice response system. See [link / reference] Figure 1 , Figure 1 This is an architecture diagram of a voice response system according to an embodiment of this application.

[0207] like Figure 1As shown, the voice response system provided in this embodiment includes a client 100 and a server 200. The client 100 includes a terminal processing unit 101, a microphone 102, a broadcaster 103, a display screen 104, and a camera 105. The terminal processing unit 101 processes data collected by the microphone 102 and the camera 105, and controls the playback and display of the broadcaster 103 and the display screen 104. The terminal processing unit 101 can also communicate with the server 200, using the server 200 to further process the data collected by the microphone 102 and the camera 105. The server 200 includes a text processing module 201 and a language response module 202. The text processing module 201 classifies the text data sent by the client 100. The text processing module 201 can also be configured with a communication protocol with the client 100, and the server 200 communicates with the client 100 using the text processing module 201. The language response module 202 stores or connects to a language model. The language response module 202 can perform semantic understanding of text data and generate response text data through the language module.

[0208] It should be noted that the language response module 202 and the text processing module 201 in this application may be located on the same server or on different servers. This application embodiment does not limit this.

[0209] The above describes the architecture diagram of the voice response system provided in the embodiments of this application. The voice response method applied to the voice response system in the embodiments of this application is described below.

[0210] like Figure 2 As shown, the voice response method provided in this application includes at least 2001-2006.

[0211] 2001: The terminal processing unit obtains the text data corresponding to the voice data collected by the microphone and sends the text data to the server.

[0212] It should be noted that the decoding of voice data in this application embodiment can be performed by the client or the server, and this application embodiment does not limit this. In one embodiment of this application, the terminal processing unit decodes the voice data to obtain the corresponding text data. As another possible implementation, the terminal processing unit can also upload the voice data to a backend server (not shown in the figure), where the voice-to-text module of the backend server decodes the voice data, and the terminal processing unit obtains the text data returned by the backend server.

[0213] In practical applications, when decoding voice data, the terminal processing unit can simultaneously generate a language identifier indicating the language of the voice data. That is, the text data in this application can include language identifiers. For example, if the client of this application supports recognizing Chinese, English, and Arabic, then the language identifiers generated by the terminal processing unit include language identifier A for English, language identifier B for Chinese, and language identifier C for Arabic. When the language corresponding to the voice data is English, the text data obtained by the terminal processing unit includes language identifier A for English.

[0214] like Figure 3 As shown, after obtaining the text data corresponding to the voice data, the client can send the text data to the server so that the server can perform further processing on the text data. In practical applications, the client can also include a camera. The camera is used to capture user image data, which assists the terminal processing unit in processing the voice data. Specifically, the terminal processing unit can extract key voice data from the voice data captured by the microphone based on the user image data; based on the key voice data, the terminal processing unit obtains the corresponding text data. In this way, the terminal processing unit can extract key voice data from the voice data captured by the microphone, reducing the computing power consumed by subsequent decoding of the voice data.

[0215] In this embodiment of the application, the microphone and camera collect data from objects that emit voice information, such as users or devices, where the devices have voice playback capabilities, such as smartphones or robots.

[0216] 2002: The server classifies the text data and obtains the classification results, which indicate the attributes of the text data.

[0217] In this embodiment, the server may include a text processing module. The text processing module can obtain classification results based on the attributes of the text data. In this embodiment, the attributes of the text data may include question attributes or instruction attributes. When receiving text data with an instruction attribute, the client needs to execute the corresponding command; for example, the text data "Let's dance" has an instruction attribute. When receiving text data with a question attribute, the client needs to answer the user's question; for example, the text data "Where is the restroom?" has a question attribute. In practical applications, the text processing module can classify the attributes of text data using keyword recognition, a relatively simple language model, or other methods; this embodiment does not limit this approach. For example, the text processing module may store a series of keywords; when text data matches this series of keywords, the text data is determined to have an instruction attribute.

[0218] 2003: Based on the classification results, the server executes the processing logic corresponding to the classification results on the text data, obtains the processing results corresponding to the attributes of the text data, and sends the processing results to the terminal processing unit.

[0219] It should be noted that the mapping relationship between the classification results and processing logic in this application embodiment can be pre-stored on the server side. The server side can determine the processing logic corresponding to the classification result based on the determined classification result and the mapping relationship table between the classification result and the processing logic. Specific examples of the classification results and processing logic in this application can be found in steps 301-304 below, and will not be repeated here.

[0220] In this embodiment, the server can classify text data of different categories based on the classification results to obtain processing results. The different categories of text data indicated by the classification results in this embodiment include text data with instruction attributes and text data with question attributes. The server can classify text data with instruction attributes and text data with question attributes based on their attributes to obtain processing results. By classifying the text data before processing, the server can better identify and process the text data in a targeted manner, thereby accelerating the response speed of the voice response system to customers while maintaining the quality of the voice response method of this application.

[0221] 2004: The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content based on the processing results.

[0222] In this embodiment, the terminal processing unit controls the loudspeaker to play audio corresponding to the processing result based on the processing result, and the display screen plays display content corresponding to the processing result. For example... Figure 3 As shown below, steps 2003-2004 in the embodiments of this application will be described in detail through steps 301-304.

[0223] 301: If the classification result indicates that the text data is an instruction attribute, the server will send the response instruction corresponding to the text data to the terminal processing unit.

[0224] like Figure 4As shown, if the classification result indicates that the text data is an instruction attribute, the server-side text processing module sends the corresponding response instruction to the terminal processing unit. The server-side text processing module can store the correspondence between keywords and response instructions. Based on this correspondence, the text processing module can send the response instruction corresponding to the keyword matching the text data to the terminal processing unit. For example, the text processing module stores keywords A, B, and C, as well as response instructions A corresponding to keyword A, B corresponding to keyword B, and C corresponding to keyword C. When the text data received by the text processing module matches keyword B, the text processing module can send the response instruction B corresponding to keyword B to the terminal processing unit.

[0225] 302: The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content according to the response command.

[0226] like Figure 4 As shown, the terminal processing unit can, based on a response command, invoke the audio corresponding to the response command and play it through a speaker. The terminal processing unit can also invoke the display content corresponding to the response command and play it through the display screen. It should be noted that the terminal processing unit can store a mapping relationship between response commands and audio, and also a mapping relationship between response commands and display content. When the terminal processing unit receives a response command, it invokes and plays the corresponding audio and display content. For example, when the terminal processing unit receives response command A, it invokes and plays the audio A and display content A corresponding to response command A.

[0227] The text data generated by the terminal processing unit based on the voice data may include a language identifier, which indicates the language of the voice data. Therefore, the response instructions generated from the text data may also include the language identifier. The terminal processing unit can select and display content or play audio corresponding to the language based on the language identifier, allowing users of different languages ​​to enjoy different experiences. For example, when the language identifier indicates Arabic, the display screen can show an interactive object matching Arabic to communicate with the user. The terminal processing unit can also display the language corresponding to the language identifier on the display screen through images or text to remind the user of the language used by the currently interacting object.

[0228] 303: If the classification result indicates that the text data is a question attribute, the server will input the text data into the language model to obtain the response text data.

[0229] The server-side in this embodiment includes a language response module, which stores or connects to a language model. It should be noted that the language module in this application can be stored within the language response module or connected to a language module; this embodiment does not impose any limitation on this. Figure 4 As shown, if the classification result indicates that the text data is a question attribute, the text processing module sends the text data to the language response module. The language response module inputs the text data into the language model to obtain the response text data. Then, the language response module can send the response text data to the terminal processing unit through the text processing module.

[0230] It should be noted that, in order to improve the response speed of the voice response method of this application, the text processing module in the aforementioned 2002 typically classifies text data using keyword matching, a relatively simple language module, or other faster classification methods. Therefore, while ensuring speed, the text processing module may identify some text data with instruction attributes as question attributes. Although the language model in this application has a slower response speed than the text processing module, it has a better understanding and judgment ability for text data. Therefore, in some possible embodiments, after the text processing module performs a first classification process, the language model will perform a second classification process on the received text data, thereby increasing the accuracy of text data classification while maintaining response speed. Specifically, the language response module uses the language model to perform semantic understanding of the text data and obtain an understanding result. If the understanding result indicates that the text data is an instruction attribute, the language response module sends the instruction corresponding to the text data to the terminal processing unit through the text processing module. If the understanding result indicates that the text data is a question attribute, the language model generates response text data based on the text data, and the language response module sends the response text data to the terminal processing unit through the text processing module.

[0231] As one possible implementation, the text data in this embodiment may include language identifiers in addition to text content. Language identifiers can assist the terminal processing unit in selecting display content or playing audio corresponding to the user's language. In this embodiment, the auxiliary language model can also assist in semantic understanding of the text data.

[0232] In most cases, large language models that support multiple languages ​​simultaneously are quite large, and their understanding and response speed to text data is relatively slow. For example, the number of parameters in current large language models supporting multiple languages ​​is typically in the hundreds of billions, making them extremely large. Therefore, in order to improve the response speed of the language model as much as possible, this embodiment divides the language model into N relatively independent sub-language models based on the language. Each sub-language model can support one language or a few languages, and its number of parameters can be in the tens of billions. Compared with single-language models that support multiple languages ​​simultaneously, the sub-language models supporting one or a few languages ​​in this embodiment are smaller and respond to text data much faster. Specifically, in a practical application, in order to improve the response speed of the voice response method as much as possible, the language module provided in this embodiment may include N sub-language models, where N is an integer greater than 1, and each of the N sub-language models corresponds to one or more language types.

[0233] The language response module can input text data into the sub-language model corresponding to the language identifier of the text data among N sub-language models to obtain response text data. As one possible implementation, the language response module can select the sub-language model matching the language identifier from the N sub-language models based on the language identifier in the text data, thereby inputting the text data into the corresponding language sub-language model to obtain response text data. The language corresponding to the response text data is consistent with the language indicated by the language identifier of the text data.

[0234] For example, the language model in this application includes sub-language model A, sub-language model B, and sub-language model C, where sub-language model A supports English, sub-language model B supports Chinese, and sub-language model C supports Arabic. The language identifiers in this application include language identifier A, language identifier B, and language identifier C, where language identifier A indicates English, language identifier B indicates Chinese, and language identifier C indicates Arabic. When the user outputs "hello!", the terminal processing unit obtains text data containing language identifier A based on the voice data collected by the microphone. After receiving the text data containing language identifier A, the language model inputs the text data into sub-language model A to obtain response text data. The response text data may also include language identifier A.

[0235] 304: The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content based on the reply text data.

[0236] In the embodiments of this application, such as Figure 4As shown, the terminal processing unit can obtain the corresponding voice data based on the reply text data. The terminal processing unit controls the broadcaster to play the reply voice data. It should be noted that the process of converting text data to voice data in this embodiment can be completed by the client or the server, and this embodiment does not limit this. In one embodiment of this application, the terminal processing unit converts text data into voice data. As another possible implementation, the terminal processing unit can also upload the text data to the backend server, where the text-to-speech module of the backend server processes the voice data, and the terminal processing unit obtains the voice data returned by the backend server.

[0237] In practical applications, the display screen can show interactive objects that interact with the user. The terminal processing unit can control the display screen to show content corresponding to the reply text data based on the reply text data. As an example, the terminal processing unit in this application can control the display screen to show the text corresponding to the reply text data and control the interactive objects on the display screen to interact with the user based on the reply text data. For example, it can control the lip movements or gestures of the interactive objects to match the audio played by the announcer, thereby facilitating the user's understanding of the reply content from multiple perspectives and improving the user experience. The reply text data in this application may include a language identifier. The terminal processing unit can also display the language type corresponding to the language identifier on the display screen through images or text to remind the user of the language type currently used by the interactive object.

[0238] To better understand the voice response method provided in the embodiments of this application, this application also provides an embodiment to facilitate understanding of the voice response method in this application.

[0239] like Figure 5As shown, after the user outputs voice, the microphone 102 collects the voice data and sends it to the terminal processing unit 101. Correspondingly, the camera 105 can also collect user image data and send it to the terminal processing unit 101. The terminal processing unit 101 extracts the voice data based on the user image data to obtain key voice data, and decodes the key voice data to obtain the corresponding text data. As an example, the terminal processing unit 101 can determine that the user is interacting with the client when the user image data indicates that the user's eyes are looking at the client, or when the user's face is facing the client, and extract the voice data output by the user at this time to obtain key voice data. When the user is not looking at the client, or when the user's face is not facing the client, it is determined that the user is not interacting with the client at this time, and the voice data output by the user at this time is non-key voice data, which does not need to be further recognized. The text data contains the text content corresponding to the voice data and the language identifier corresponding to the voice data. The terminal processing unit 101 sends the obtained text data to the text processing module 201, which classifies the text data to obtain the classification result.

[0240] If the classification result indicates that the text data is an instruction attribute, the text processing module 201 sends the corresponding response instruction to the terminal processing unit 101. The response instruction also includes a language identifier. The terminal processing unit controls the loudspeaker to play the audio corresponding to the response instruction and controls the display screen to play the display content corresponding to the response instruction. The language used in the audio corresponding to the response instruction can match the language identifier. The image of the interactive object in the display content can match the language identifier, and the display content can also include language-specific display elements, which represent the language identifier through images or text.

[0241] If the classification result indicates that the text data is a question attribute, the text processing module 201 sends the text data to the language response module 202. The language response module 202 selects the sub-language model corresponding to the language identifier from among N sub-language models in the language model based on the language identifier in the text data, and inputs the text data into the sub-language model corresponding to the language identifier to obtain the response text data. The response text data may still contain the language identifier. The language response module 202 sends the response text data to the terminal processing unit 101 through the text processing module 201. The terminal processing unit 101 controls the loudspeaker to play the audio corresponding to the response text data based on the response text data. The terminal processing unit 101 can control the interactive object on the display screen to display the image corresponding to the language identifier, and control the gestures and lip movements of the interactive object to match the audio played by the loudspeaker. The terminal processing unit 101 can also display the language type corresponding to the language identifier on the display screen through images or text to remind the user of the language type currently used by the interactive object. For example, the display screen shows display elements corresponding to multiple language identifiers, including language identifier 1. When the display screen shows text data 1 in the language corresponding to language identifier 1, and the broadcaster plays the audio of text data 1 in the language corresponding to language identifier 1, the element corresponding to language identifier 1 is highlighted. In one example, the highlighting method is flashing; in another example, the highlighting method is that the element corresponding to language identifier 1 is framed by a focus box.

[0242] In summary, the voice response method provided in this application converts voice data into text data. The server then categorizes the attributes of the text data. If the text data is an instruction, the corresponding response instruction is sent to the client. If the text data is a question, it is input into a language model to obtain the response text data. The client then displays or plays the response text data accordingly. Thus, by pre-categorizing the text data, the voice response method provided in this application avoids further recognition of instruction-type data through a language model, thereby accelerating the response speed of the voice response system while maintaining the quality of the answer.

[0243] Based on the above-described voice response method, this application also provides another voice response method, which is applied to a client-side communication connection with a server. The client includes a microphone, a speaker, a display screen, and a terminal processing unit, such as... Figure 6 As shown, the method includes 601-603.

[0244] 601: The terminal processing unit obtains the text data corresponding to the voice data collected by the microphone.

[0245] 602: The terminal processing unit sends text data to the server so that the server can classify the text data and obtain the classification result. The classification result indicates the attributes of the text data. The classification result is used to assist the server in executing the processing logic corresponding to the classification result on the text data and obtaining the processing result corresponding to the attributes of the text data.

[0246] 603: The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content based on the processing results.

[0247] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0248] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0249] In some possible embodiments, the server side includes a text processing module;

[0250] The text processing module is configured to classify text data and obtain classification results;

[0251] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0252] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0253] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module;

[0254] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0255] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0256] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0257] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0258] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0259] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0260] In some possible embodiments, the terminal processing unit controls the speaker to play audio and / or the display screen to display content based on the reply text data, including:

[0261] The terminal processing unit obtains the corresponding voice data based on the response text data.

[0262] The terminal processing unit controls the broadcaster to play the response voice data.

[0263] In some possible embodiments, the terminal processing unit obtains text data corresponding to the voice data collected by the microphone, including:

[0264] The terminal processing unit decodes the voice data to obtain the corresponding text data.

[0265] Alternatively, the terminal processing unit uploads the voice data to the backend server for decoding, and obtains the text data returned by the backend server.

[0266] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit controls the display screen to play displayed content based on the reply text data, including:

[0267] The terminal processing unit controls the display screen to show the text corresponding to the reply text data based on the reply text data, and controls the interactive objects displayed on the display screen to interact with the user based on the reply text data.

[0268] In some possible embodiments, the client may also include a camera;

[0269] The camera is used to collect user image data;

[0270] The terminal processing unit obtains the corresponding text data based on the voice data collected by the microphone, including:

[0271] The terminal processing unit extracts key voice data from the microphone based on the user's image data.

[0272] The terminal processing unit obtains the text data corresponding to the key voice data based on the key voice data.

[0273] It should be noted that the voice response method in the embodiments of this application can implement the various processes of the aforementioned voice response method embodiments and achieve the same effect and function, which will not be repeated here.

[0274] Based on the above-described voice response method, this application also provides a voice response method applied to a server, wherein the server and client are connected in communication, and the client includes a microphone, a speaker, a display screen, and a terminal processing unit, such as... Figure 7 As shown, the method includes 701-704.

[0275] 701: The server receives text data sent by the terminal processing unit. The text data is generated based on the voice data collected by the radio.

[0276] 702: The server classifies the text data and obtains the classification results, which indicate the attributes of the text data.

[0277] 703: The server executes the processing logic corresponding to the classification result on the text data to obtain the processing result corresponding to the attributes of the text data.

[0278] 704: The server sends the processing result to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play displayed content based on the processing result.

[0279] In some possible embodiments, the processing result includes a response instruction. The server executes processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0280] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit.

[0281] In some possible embodiments, the processing result includes response text data. The server performs processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including:

[0282] If the classification result indicates that the text data is a question attribute, the server will input the text data into the language model to obtain the response text data.

[0283] In some possible embodiments, the server side includes a text processing module;

[0284] The server classifies the text data and obtains the classification results, including:

[0285] The text processing module classifies the text data and obtains the classification results;

[0286] If the classification result indicates that the text data is an instruction attribute, the server will send the corresponding response instruction to the terminal processing unit, including:

[0287] If the classification result indicates that the text data is an instruction attribute, the text processing module will send the instruction corresponding to the text data to the terminal processing unit.

[0288] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0289] If the classification result indicates that the text data is a question attribute, the server inputs the text data into the language model to obtain the response text data, including:

[0290] If the classification result indicates that the text data is a question attribute, the text processing module sends the text data to the language response module;

[0291] The language response module inputs text data into the language model, obtains response text data, and sends the response text data to the terminal processing unit through the text processing module.

[0292] In some possible embodiments, it also includes:

[0293] The language response module uses a language model to perform semantic understanding of text data and obtain the understanding results.

[0294] If the interpretation result indicates that the text data is an instruction attribute, the language response module will send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0295] If the interpretation result indicates that the text data is a question attribute, the language response module generates the corresponding response text data through the language model, and sends the response text data to the terminal processing unit through the text processing module.

[0296] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0297] The language module includes one or more sub-language models, each of which corresponds to one or more language categories. The language response module inputs text data into the language model to obtain response text data, including:

[0298] The language response module inputs text data into one or more sub-language models corresponding to the language identifier, and obtains the response text data corresponding to the language identifier.

[0299] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0300] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0301] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0302] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0303] In some possible embodiments, the client may also include a camera;

[0304] The camera is used to collect user image data;

[0305] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone, including:

[0306] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0307] It should be noted that the voice response method in the embodiments of this application can implement the various processes of the aforementioned voice response method embodiments and achieve the same effect and function, which will not be repeated here.

[0308] In the description of this specification, references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this application, and the aforementioned terms do not necessarily refer to the same implementation or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0309] The method flowcharts for embodiments of this application describe certain operations as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, or some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., by software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0310] Those skilled in the art will understand that in the methods described in the above specific embodiments, the order in which the steps are written does not imply a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.

[0311] Based on the above-described voice response method, this application also provides a voice response system, such as... Figure 8 As shown, the voice response system includes a client 100 and a server 200. The client 100 includes a microphone 102, a loudspeaker 103, a display screen 104, and a terminal processing unit 101.

[0312] The terminal processing unit 101 is configured to obtain text data corresponding to the voice data collected by the microphone and send the text data to the server 200.

[0313] The server-side 200 is configured to classify text data, obtain classification results, and the classification results indicate the attributes of the text data;

[0314] The server 200 is configured to perform processing logic corresponding to the classification result on the text data according to the classification result, obtain the processing result corresponding to the attribute of the text data, and send the processing result to the terminal processing unit 100.

[0315] The terminal processing unit 100 is configured to control the speaker 103 to play audio and / or the display screen 104 to play displayed content based on the processing result.

[0316] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0317] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0318] In some possible embodiments, the server side includes a text processing module;

[0319] The text processing module is configured to classify text data and obtain classification results;

[0320] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0321] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0322] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module.

[0323] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0324] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0325] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0326] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0327] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0328] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0329] In some possible embodiments, the terminal processing unit is configured to obtain the reply voice data corresponding to the reply text data based on the reply text data;

[0330] The terminal processing unit is configured to control the broadcaster to play back the response voice data.

[0331] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0332] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0333] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0334] In some possible embodiments, the client may also include a camera;

[0335] The camera is used to collect user image data;

[0336] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0337] It should be noted that the voice response system in this application embodiment can implement the various functions of the aforementioned voice response method embodiment and achieve the same effect, which will not be repeated here.

[0338] Based on the aforementioned voice response system, this application provides a client 100, such as... Figure 9 As shown, the client 100 is connected to the server for communication. The client 100 includes a microphone 102, a loudspeaker 103, a display screen 104, and a terminal processing unit 101.

[0339] The terminal processing unit 101 is configured to obtain text data corresponding to the voice data collected by the microphone.

[0340] The terminal processing unit 101 is configured to send text data to the server so that the server can classify the text data and obtain the classification result, the classification result indicating the attributes of the text data.

[0341] The classification results are configured to assist the server in performing the processing logic corresponding to the classification results on the text data, and to obtain the processing results corresponding to the attributes of the text data.

[0342] The terminal processing unit 101 is configured to control the speaker 102 to play audio and / or the display screen 104 to play displayed content based on the processing result.

[0343] In some possible embodiments, the processing result includes a response instruction. If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0344] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0345] In some possible embodiments, the server side includes a text processing module;

[0346] The text processing module is configured to classify text data and obtain classification results;

[0347] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0348] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0349] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module;

[0350] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0351] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0352] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0353] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0354] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0355] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0356] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0357] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0358] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0359] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0360] In some possible embodiments, the client may also include a camera;

[0361] The camera is used to collect user image data;

[0362] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0363] It should be noted that the client in this application embodiment can implement the various functions of the aforementioned voice response system embodiment and achieve the same effect, which will not be repeated here.

[0364] Based on the aforementioned voice response system, this application provides a server-side component, such as... Figure 10As shown, the server 200 is connected to the client 100 in communication. The client 100 includes a radio, a loudspeaker, a display screen, and a terminal processing unit.

[0365] The server 200 is configured to receive text data sent by the terminal processing unit, and the text data is generated based on the voice data collected by the radio.

[0366] The server-side 200 is configured to classify text data, obtain classification results, and the classification results indicate the attributes of the text data;

[0367] The server-side 200 is configured to perform processing logic corresponding to the classification results on the text data based on the classification results, and obtain the processing results corresponding to the attributes of the text data.

[0368] The server 200 is configured to send processing results to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play displayed content based on the processing results.

[0369] In some possible embodiments, the processing result includes a response command;

[0370] If the classification result indicates that the text data is an instruction attribute, the server is configured to send the response instruction corresponding to the text data to the terminal processing unit.

[0371] In some possible embodiments, the processing result includes response text data. If the classification result indicates that the text data is a question attribute, the server is configured to input the text data into a language model to obtain response text data.

[0372] In some possible embodiments, the server side includes a text processing module;

[0373] The server is configured to classify text data and obtain classification results, including:

[0374] The text processing module is configured to classify text data and obtain classification results;

[0375] If the classification result indicates that the text data is an instruction attribute, the text processing module is configured to send the instruction corresponding to the text data to the terminal processing unit.

[0376] In some possible embodiments, the server side also includes a language response module, which includes or is connected to a language model;

[0377] If the classification result indicates that the text data is a question attribute, the text processing module is configured to send the text data to the language response module.

[0378] The language response module is configured to input text data into the language model, obtain response text data, and send the response text data to the terminal processing unit through the text processing module.

[0379] In some possible embodiments, the language response module is configured to perform semantic understanding of the text data through a language model to obtain the understanding result;

[0380] If the interpretation result indicates that the text data is an instruction attribute, the language response module is configured to send the instruction corresponding to the text data to the terminal processing unit through the text processing module.

[0381] If the understanding result indicates that the text data is a question attribute, the language response module is configured to generate the response text data corresponding to the text data through the language model, and send the response text data to the terminal processing unit through the text processing module.

[0382] In some possible embodiments, the text data includes a language identifier that indicates the language of the speech data;

[0383] The language module includes one or more sub-language models, each of which corresponds to one or more language types. The language response module is configured to input text data into the sub-language model corresponding to the language identifier in one or more sub-language models, and obtain the response text data corresponding to the language identifier.

[0384] In some possible embodiments, the terminal processing unit is configured to obtain the corresponding reply voice data based on the reply text data, and control the broadcaster to play the reply voice data.

[0385] In some possible embodiments, the terminal processing unit is configured to decode the voice data to obtain the text data corresponding to the voice data;

[0386] Alternatively, the terminal processing unit is configured to upload voice data to a backend server for decoding, and obtain text data returned by the backend server.

[0387] In some possible embodiments, the display screen is used to display interactive objects, and the terminal processing unit is configured to control the display screen to display the text corresponding to the reply text data according to the reply text data, and to control the interactive objects displayed on the display screen to interact with the user according to the reply text data.

[0388] In some possible embodiments, the client may also include a camera;

[0389] The camera is used to collect user image data;

[0390] The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone, including:

[0391] The terminal processing unit is configured to extract key voice data from the microphone based on the user's image data, and then obtain the corresponding text data based on the key voice data.

[0392] It should be noted that the server in this application embodiment can implement the various functions of the aforementioned voice response system embodiment and achieve the same effect, which will not be repeated here.

[0393] According to some embodiments of this application, this application provides a client-side non-volatile computer storage medium storing computer-executable instructions configured to execute, when run by a processor, the actions performed by the client in the voice response method described in the above embodiments. According to some embodiments of this application, this application also provides a server-side non-volatile computer storage medium storing computer-executable instructions configured to execute, when run by a processor, the actions performed by the server in the voice response method described in the above embodiments.

[0394] According to some embodiments of this application, this application provides a computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to execute the aforementioned voice response method applied on a client side. This application also provides a computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to execute the aforementioned voice response method applied on a server side. Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally, certain steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple sub-steps.

[0395] While the spirit and principles of this application have been described above with reference to several specific embodiments, it should be understood that this application is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined. This application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A voice response method, characterized in that, The method is applied to a voice response system, which includes a client and a server. The client includes a microphone, a speaker, a display screen, and a terminal processing unit. The method includes: The terminal processing unit obtains the text data corresponding to the voice data collected by the microphone and sends the text data to the server. The server classifies the text data and obtains a classification result, which indicates the attributes of the text data. The server performs processing logic corresponding to the classification result on the text data according to the classification result, obtains the processing result corresponding to the attribute of the text data, and sends the processing result to the terminal processing unit; The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content based on the processing result.

2. The method according to claim 1, characterized in that, The processing result includes a response instruction. The server executes processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including: If the classification result indicates that the text data is an instruction attribute, the server will send the response instruction corresponding to the text data to the terminal processing unit.

3. The method according to claim 1, characterized in that, The processing result includes response text data. The server executes processing logic corresponding to the classification result on the text data based on the classification result to obtain the processing result, including: If the classification result indicates that the text data is a question attribute, the server inputs the text data into the language model to obtain the response text data.

4. The method according to claim 2, characterized in that, The server-side includes a text processing module; The server classifies the text data to obtain classification results, including: The text processing module classifies the text data and obtains the classification results; If the classification result indicates that the text data is an instruction attribute, the server sends the response instruction corresponding to the text data to the terminal processing unit, including: If the classification result indicates that the text data is an instruction attribute, the text processing module sends the instruction corresponding to the text data to the terminal processing unit.

5. The method according to claim 3, characterized in that, The server-side also includes a language response module, which includes or is connected to a language model; If the classification result indicates that the text data is a question attribute, the server inputs the text data into the language model to obtain response text data, including: If the classification result indicates that the text data is a question attribute, the text processing module will send the text data to the language response module; The language response module inputs the text data into the language model to obtain response text data, and then sends the response text data to the terminal processing unit through the text processing module.

6. A voice response method, characterized in that, The method is applied to a client, which communicates with a server. The client includes a microphone, a loudspeaker, a display screen, and a terminal processing unit. The method includes: The terminal processing unit obtains the text data corresponding to the voice data collected by the microphone. The terminal processing unit sends the text data to the server so that the server can classify the text data and obtain a classification result, the classification result indicating the attributes of the text data. The classification result is used to assist the server in performing processing logic corresponding to the classification result on the text data to obtain processing results corresponding to the attributes of the text data; The terminal processing unit controls the speaker to play audio and / or the display screen to play displayed content based on the processing result.

7. A voice response method, characterized in that, The method is applied to a server, which communicates with a client. The client includes a microphone, a loudspeaker, a display screen, and a terminal processing unit. The method includes: The server receives text data sent by the terminal processing unit, and the text data is generated based on the voice data collected by the microphone. The server classifies the text data and obtains a classification result, which indicates the attributes of the text data. The server executes processing logic corresponding to the classification result on the text data according to the classification result, and obtains processing results corresponding to the attributes of the text data; The server sends the processing result to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play content based on the processing result.

8. A voice response system, characterized in that, The voice response system includes a client and a server. The client includes a microphone, a speaker, a display screen, and a terminal processing unit. The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone, and send the text data to the server. The server is configured to classify the text data and obtain a classification result, wherein the classification result indicates the attributes of the text data; The server is configured to perform processing logic corresponding to the classification result on the text data according to the classification result, obtain processing result corresponding to the attribute of the text data, and send the processing result to the terminal processing unit; The terminal processing unit is configured to control the speaker to play audio and / or the display screen to play displayed content based on the processing result.

9. A client, characterized in that, The client communicates with the server, and the client includes a microphone, a loudspeaker, a display screen, and a terminal processing unit. The terminal processing unit is configured to obtain text data corresponding to the voice data collected by the microphone. The terminal processing unit is configured to send the text data to the server so that the server can classify the text data and obtain a classification result, wherein the classification result indicates the attributes of the text data. The classification result is configured to assist the server in performing processing logic corresponding to the classification result on the text data, so as to obtain processing results corresponding to the attributes of the text data; The terminal processing unit is configured to control the speaker to play audio and / or the display screen to play displayed content based on the processing result.

10. A server-side component, characterized in that, The server and the client are connected in communication. The client includes a microphone, a loudspeaker, a display screen, and a terminal processing unit. The server is configured to receive text data sent by the terminal processing unit, the text data being generated based on the voice data collected by the microphone; The server is configured to classify the text data and obtain a classification result, wherein the classification result indicates the attributes of the text data; The server is configured to perform processing logic corresponding to the classification result on the text data according to the classification result, and obtain processing results corresponding to the attributes of the text data; The server is configured to send the processing result to the terminal processing unit so that the terminal processing unit can control the speaker to play audio and / or the display screen to play content based on the processing result.

Images