Information interaction method and device, storage medium and program product

By installing AI voice assistants on the terminal device and connecting them with customer service programs, the voice recognition module is used to solve the problem of inconvenient operation of disabled customer service personnel, improving interaction efficiency and user experience.

CN119025203AActive Publication Date: 2024-11-26ZHEJIANG TMALL TECH CO LTD

Patent Information

Application Number
CN202410997326.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-26
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

Disabled customer service personnel cannot flexibly use keyboards and mouse due to physical disorders, resulting in low interaction efficiency and low user satisfaction.

Method used

Install an AI voice assistant on the terminal device, associate it with the customer service program, receive instructions and execute it through the voice recognition module, output result information, assist customer service personnel in completing interaction, and solve the problem of inconvenience in manual input.

Benefits of technology

It improves interaction efficiency and is suitable for various scenarios, especially interactive scenarios where people with disabilities participate, improving user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025203A_ABST
    Figure CN119025203A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an information interaction method and device, a storage medium and a program product. In the method, an AI voice assistant associated with a customer service program is provided for customer service personnel, and on one hand, the AI voice assistant is responsible for displaying a customer service interface provided by the customer service program; on the other hand, a voice interaction instruction of the customer service staff can be responded and executed to give out execution result information, content capable of serving as reply information can be selected from the execution result information to be displayed in a question and answer area of the customer service interface according to the voice instruction of the user needing to be replied, the customer service staff is assisted to complete interaction with the user, and in the process, the user experience is improved. The customer service staff can complete interaction through simple voice input under the assistance of the AI voice assistant, the problem that peripheral equipment cannot be flexibly used for interaction due to inconvenience in manual input can be solved, and the problem of low interaction efficiency caused by the problem can be solved. The scheme is suitable for various interaction scenes, especially interaction scenes in which the handicapped participates.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an information interaction method, device, storage medium and program product. Background Art

[0002] At present, some companies have launched a new customer service solution, namely cloud customer service based on cloud computing technology, to help solve the employment problems of people with disabilities. Cloud customer service can work anywhere with Internet access, and can create an accessible work platform for people with disabilities, allowing them to provide professional customer service to corporate users at home or even in bed through computers or smartphones.

[0003] Customer service personnel access the customer service interface through terminal devices such as computers or smart phones, understand the user's problem information on the customer service interface, and interact with the terminal device through peripherals such as keyboards and mice to answer various questions for users, such as helping users to query order information or logistics information. However, due to various physical obstacles, disabled customer service personnel may not be able to flexibly use peripherals such as keyboards and mice, so they have inconvenient operations, resulting in low customer service efficiency and low user satisfaction. Summary of the invention

[0004] Multiple aspects of the present application provide an information interaction method, device, storage medium and program product to provide an efficient and fast interaction method to solve the problem of low interaction efficiency in various interaction scenarios, especially interaction scenarios involving disabled people.

[0005] An embodiment of the present application provides an information interaction method, which is applied to an AI voice assistant on a terminal device, wherein the AI ​​voice assistant is associated with a customer service program on the terminal device, and the method comprises: displaying a customer service interface provided by the customer service program, wherein the customer service interface comprises a question and answer area, and the question and answer area is used to display at least one target question information for a target user to consult a customer service personnel; based on a voice recognition module in the AI ​​voice assistant, receiving at least one voice interaction instruction initiated by the customer service personnel for any target question information; for any voice interaction instruction, executing the any voice interaction instruction, and outputting the execution result information of the any voice interaction instruction in a display and / or voice manner; for the target voice interaction instruction in the at least one voice interaction instruction, selecting at least part of the execution result information of the target voice interaction instruction as the target reply information of the any target question information and displaying it in the question and answer area.

[0006] An embodiment of the present application also provides an information interaction method, which is applied to an AI voice assistant on a terminal device, and the AI ​​voice assistant is associated with a target program on the terminal device. The method includes: displaying an interaction interface provided by the target program, the interaction interface including an interaction area, and the interaction area is used to display at least one interaction information initiated by a first user to a second user; based on a voice recognition module in the AI ​​voice assistant, receiving at least one voice interaction instruction initiated by the second user for any interaction information; for any voice interaction instruction, executing the any voice interaction instruction, and outputting the execution result information of the any voice interaction instruction in a display and / or voice manner; for a target voice interaction instruction in the at least one voice interaction instruction, selecting at least part of the execution result information of the target voice interaction instruction as target reply information of the any interaction information and displaying it in the interaction area.

[0007] An embodiment of the present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory to implement the steps in the information interaction method.

[0008] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps of any one of the methods described in the information interaction method.

[0009] An embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the steps in the information interaction method.

[0010] In this embodiment, an AI voice assistant is provided for customer service personnel. The AI ​​voice assistant is associated with the customer service program. On the one hand, it is responsible for displaying the customer service interface provided by the customer service program; on the other hand, it provides voice functions for customer service personnel, can respond to and execute the voice interaction instructions of customer service personnel, and then give the execution result information. For the voice instructions that need to be replied to by the user, it can select the content that can be used as the reply information from the corresponding execution result information and display it in the question and answer area, which can assist the customer service personnel to complete the interaction with the user to a certain extent. In this process, the customer service personnel can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of inability to flexibly use peripherals such as mouse and keyboard due to the inconvenience of manual input, and then solve the problem of low interaction efficiency caused by this. This solution has strong universality and is suitable for various interaction scenarios, especially interaction scenarios involving disabled people.

[0011] The embodiment of the present application also provides an information interaction method, which provides an AI voice assistant for users. The AI ​​voice assistant is associated with a target program for interaction between users. On the one hand, it is responsible for displaying the interaction interface provided by the target program; on the other hand, it provides voice functions for users, can respond to and execute the user's voice interaction instructions and then give execution result information, and for voice instructions that need to be replied to another user, can select content that can be used as reply information from the corresponding execution result information and display it on the interaction interface, which can assist users in interaction to a certain extent. In this process, users can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of not being able to flexibly use peripherals such as mouse and keyboard for interaction due to the inconvenience of manual input, and thus solve the problem of low interaction efficiency caused by this. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0013] Figure 1a A flowchart of an information interaction method provided by an exemplary embodiment of the present application;

[0014] Figure 1b A schematic diagram of a customer service interface provided for an exemplary embodiment of the present application;

[0015] Figure 2 A schematic diagram of displaying an interactive interface in a floating window form provided by an exemplary embodiment of the present application;

[0016] Figure 3 A schematic diagram of an interface state of an interactive interface in a hidden state provided by an exemplary embodiment of the present application;

[0017] Figure 4 A schematic diagram of the interface state for waking up an AI voice assistant provided by an exemplary embodiment of the present application;

[0018] Figure 5 A schematic diagram of an interface state for displaying a refund card provided by an exemplary embodiment of the present application;

[0019] Figure 6 A schematic diagram of the interface status for selecting and sending speech techniques provided by an exemplary embodiment of the present application;

[0020] Figure 7 A schematic diagram of an interface state for voice input in a question and answer area provided by an exemplary embodiment of the present application;

[0021] Figure 8A schematic diagram of an interface state for displaying at least one user session information provided by an exemplary embodiment of the present application;

[0022] Fig. 9 A schematic diagram of the interface status before and after the session switching provided by an exemplary embodiment of the present application;

[0023] Fig.10 A flowchart of another information interaction method provided by an exemplary embodiment of the present application;

[0024] Fig.11 A schematic diagram of an information interaction device provided by an exemplary embodiment of the present application;

[0025] Fig.12 A schematic diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portals for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) are in compliance with relevant laws and standards.

[0028] In some existing customer service scenarios, there may be situations where it is impossible to flexibly use or temporarily inconvenient to use peripherals such as keyboards and mice for interaction. For example, when disabled customer service personnel interact with terminal devices to answer various questions for users, they may have various physical disabilities, such as being unable to flexibly use peripherals such as keyboards and mice, so there are inconveniences in operation, resulting in low customer service efficiency and low user satisfaction. For another example, when non-disabled customer service personnel are holding something in their hands or are occupied, they may temporarily be unable to use peripherals such as keyboards and mice, which can also lead to low customer service efficiency and low user satisfaction.

[0029] In response to the above technical problems, the embodiments of the present application provide an AI (Artificial Intelligence) voice assistant solution for customer service personnel. The AI ​​voice assistant and the customer service program are installed on the terminal device. The AI ​​voice assistant can be associated with the customer service program. Customer service personnel can make various interactive controls on the customer service interface provided by the customer service program with the assistance of the AI ​​voice assistant through simple voice input. In this way, even when it is impossible to flexibly use or inconvenient to use peripherals such as the keyboard and mouse, customer service services can still be provided to users more efficiently, thereby improving the user experience.

[0030] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0031] Figure 1a This is a flowchart of an information interaction method provided by an exemplary embodiment of the present application. The method can be executed by an AI voice assistant on a terminal device. The terminal device can be a terminal device such as a smartphone, tablet computer, computer or smart watch, and this implementation does not limit it. The terminal device can be configured with an AI voice assistant, and the AI ​​voice assistant is associated with the customer service program on the terminal device.

[0032] In the embodiments of the present application, the product implementation form of the AI ​​voice assistant and the customer service program is not limited. Among them, the AI ​​voice assistant and the customer service program can be implemented as independent program products, for example, the customer service program is a website or APP (Application), and the AI ​​voice assistant is implemented as an APP; during the use of the customer service program, the AI ​​voice assistant can be started, and the AI ​​voice assistant takes over some functions of the customer service program to assist the customer service personnel in completing information interaction, such as being responsible for displaying the customer service interface and performing relevant control operations required to assist the customer service personnel in completing information interaction on this basis. Alternatively, the AI ​​voice assistant and the customer service program can also be implemented as a program product, for example, the AI ​​voice assistant is embedded or integrated in the customer service program, and the AI ​​voice assistant can be implemented as a functional module of the customer service program to assist the customer service personnel in completing the interaction. Alternatively, the AI ​​voice assistant and the customer service program can also be implemented as independent program products that are coupled with each other, for example, the AI ​​voice assistant and the customer service program are independent program products, but an access entry to the customer service program is embedded in the AI ​​voice assistant, and the access entry can be an access link provided by the customer service program to the outside, so as to enter the customer service program through the AI ​​voice assistant, display the customer service interface provided by the customer service program, and perform relevant control operations required to assist the customer service personnel in completing information interaction on this basis.

[0033] Regardless of which of the above-mentioned product implementation methods is used, the process of AI voice assistants assisting customer service personnel in completing information interaction is the same or similar. Figure 1a As shown, the method may include the following steps:

[0034] Step 11: Display a customer service interface provided by the customer service program, wherein the customer service interface includes a question and answer area, and the question and answer area is used to display at least one target question information for the target user to consult the customer service personnel.

[0035] Step 12: Based on the voice recognition module in the AI ​​voice assistant, receive at least one voice interaction command initiated by the customer service staff for any target question information.

[0036] Step 13: execute any voice interaction instruction and output the execution result information of any voice interaction instruction in the form of display and / or voice.

[0037] Step 14: For a target voice interaction instruction in at least one voice interaction instruction, select at least part of the execution result information of the target voice interaction instruction as target reply information for any target question information and display it in the question and answer area.

[0038] In this embodiment, the AI ​​voice assistant can be used to assist customer service personnel in interactively controlling the customer service interface, and the AI ​​voice assistant may include: a non-manual recognition module. The non-manual recognition module refers to an information recognition module that does not rely on manual input, and can rely on other information input methods other than manual input for information recognition. The non-manual recognition module includes at least a speech recognition module, and the speech recognition module refers to a module that recognizes information based on speech input. Further optionally, the non-manual recognition module may also include a head movement recognition module, and the head movement recognition module refers to a module that recognizes information based on head movements or movements issued by other parts of the head (such as eye movements, mouth movements, etc.). In this embodiment, the speech recognition module is specifically used to identify the voice interaction instructions issued by the customer service personnel during the operation of the AI ​​voice assistant. The head movement recognition module is specifically used to identify the head movements issued by the customer service personnel or movements issued by other parts of the head during the operation of the AI ​​voice assistant, and then identify the instructions that the customer service personnel want to issue. In this embodiment, the AI ​​voice assistant can free customer service personnel from manual input operations through these non-manual recognition modules, allowing customer service personnel to quickly complete information input on the customer service interface through non-manual input methods such as voice and head movement, thereby improving interaction efficiency.

[0039] In this embodiment, the AI ​​voice assistant can display the customer service interface provided by the customer service program, such as Figure 1b As shown, the customer service interface includes at least a question and answer area, and may further include a function area and a conversation area. Figure 1bThe customer service interface style shown is only an example and is not limited to this. The question and answer area is used to display at least one target question information that the target user uses to consult the customer service staff. In this embodiment, the user currently responsible for the customer service staff is called the target user, and the question information consulted by the target user is called the target question information. The target question information is used to describe the questions consulted by the target user to the customer service staff. It should be noted that, depending on the customer service scenario, the target user and the target question information consulted by the target user will be different. For example: If it is an e-commerce scenario, the target user can be a consumer of the e-commerce platform, and the target question information may be question information related to goods, logistics, orders, refunds, returns, etc.; for example, the target question information may be "When can this product be shipped" or "Why is it not returned yet", etc. If it is a travel scenario, the target user may be a tourist or traveler, and the target question information may be question information related to itinerary consultation, hotel reservation, scenic spot inquiry, etc.; if it is a financial industry, the target user may be a financial management or savings user, and the target question information may be question information related to account inquiry, transaction consultation, loan application, etc.

[0040] Among them, the target user can consult the customer service staff with multiple target question information, and the multiple target question information can be independent of each other or have a correlation relationship, for example, the next target question information is a follow-up question of the previous target question information. For multiple target question information, the target user can consult the customer service staff at the same time or in the same batch, or in batches and at different times, and there is no limitation on this.

[0041] In this embodiment, the target user consults the customer service staff about the target problem information through the customer service interface displayed on the terminal device, the terminal device obtains the target problem information and sends the target problem information to the server, the server sends the target problem information to the customer service program on the terminal device of the customer service staff, and the customer service program displays the target problem information in the question and answer area of ​​the customer service interface to present it to the customer service staff. Accordingly, the customer service staff can view at least one target problem information through the question and answer area of ​​the customer service interface, and can process any target problem information. In this embodiment, the customer service staff processes the target problem information mainly refers to the process of obtaining reply information related to the target problem information and sending the reply information to the question and answer area to provide it to the target user. Among them, the way in which the customer service staff obtains the reply information related to the target problem information will be different for different target problem information, but the entire reply process requires the customer service staff to interact with the terminal device.

[0042] In this embodiment, in order to simplify the interactive operations in which customer service personnel participate, an AI voice assistant is provided. On the one hand, the customer service personnel can input information with the terminal device in a simple voice manner. On the other hand, the AI ​​voice assistant can replace the customer service personnel to process the reply information. The AI ​​voice assistant assists the customer service personnel from the information input link to the problem handling link, simplifying the interactive operations required for the customer service personnel to reply to the target problem information. Based on this, when replying to any target problem information, the customer service personnel can initiate a voice interaction instruction to the AI ​​voice assistant for any target problem information. Among them, the number of voice interaction instructions initiated by the customer service personnel for any target problem information can be one or more times. In the case of initiating multiple voice interaction instructions for any target problem information, the multiple voice interaction instructions can be initiated in sequence. Further, optionally, the multiple voice interaction instructions are all for the purpose of serving the any target problem information. The AI ​​voice assistant can receive at least one voice interaction instruction initiated by the customer service personnel for any target problem information based on the voice recognition module. The voice recognition module can call the audio module (such as a microphone) of the terminal device to receive the voice interaction instruction initiated by the customer service personnel; then, for any voice interaction instruction received, any voice interaction instruction is executed, and the execution result information of any voice interaction instruction is output. The execution result information is used to describe the result obtained after executing the voice interaction instruction of the customer service staff. In the case of multiple voice interaction instructions, the AI ​​voice assistant can receive and execute them one by one in the order of the multiple voice interaction instructions.

[0043] In this embodiment, in addition to supporting voice functions, the AI ​​voice assistant can also integrate a large language model for information processing. The large language model (LLM) is a deep learning model based on massive text data training. In the embodiment of the present application, the large language model is referred to as the target language model. Among them, the AI ​​voice assistant can use the target language model to perform corresponding natural language processing on the information sent by the customer service staff or users. For example, when executing some voice interaction instructions, it can rely on the target language model, without limitation. The target language model can be a language model developed according to a specific customer service scenario, or a language model obtained by fine-tuning an existing large language model according to a specific customer service scenario, or an existing large language model can be directly used, without limitation. Among them, the existing large language models that can be used in this embodiment include but are not limited to: LaMDA (Language Models for Dialog Applications, language model for dialogue applications) model, LLaMa (Large Language Model Family, large language model family), Qwen-Max model, etc. The detailed process of the AI ​​voice assistant executing voice interaction instructions can be found in the description of subsequent embodiments, which will not be described in detail here.

[0044] Among them, the purpose of outputting the execution result information is to assist customer service personnel in determining the target reply information of the target question information; customer service personnel can generate the reply information of the target question information by themselves based on the execution result information of any voice interaction instruction, or the AI ​​voice assistant can directly obtain the reply information of the target question information from the execution result information instead of the customer service personnel, for example, using all or part of the execution result information as the reply information of the target question information.

[0045] Among them, the form of outputting the execution result information can be a display method, that is, the execution result information is displayed, for example, it can be displayed on the current customer service interface or the interactive interface provided by the AI ​​voice assistant, which can be determined according to the type of voice interaction instruction; the form of outputting the execution result information can also be a voice method, that is, the execution result information is output in a voice broadcasting method; the form of outputting the execution result information can also be a combination of a display method and a voice method, that is, the execution result information is displayed and the execution result information is output in a voice broadcasting method. Optionally, a correspondence can be established in advance between the type of voice interaction instruction and the output form of the execution result information, based on which the specific form of outputting the execution result information can be determined according to the type of voice interaction instruction. Alternatively, optionally, it can also be defaulted to uniformly adopt a display method, a voice method, or a combination of a display method and a voice method.

[0046] In this embodiment, at least one voice interaction instruction initiated by the customer service staff for any target question information may include: voice interaction instructions that require a reply to the target user and voice interaction instructions that do not require a reply to the target user. The voice interaction instructions that require a reply to the target user are referred to as target voice interaction instructions; conversely, the voice interaction instructions that do not require a reply to the target user are referred to as non-target voice interaction instructions. For example, the target voice interaction instructions that require a reply to the target user may be: a refund instruction or a speech instruction, etc., wherein the refund instruction is used to instruct the AI ​​voice assistant to obtain the refund operation information and send the refund operation information to the target user to guide the user to initiate a refund operation; the speech instruction is used to instruct the AI ​​voice assistant to generate relevant speech information and select the speech information that matches the target question information sent by the target user as the reply information and send it to the target user. The non-target voice interaction instructions that do not require a reply to the target user may be: background data call instructions, page call instructions, search engine call instructions, or customer service program login instructions, etc. Among them, the background data call instruction can be used to call and display the data stored in the background, so that customer service personnel can refer to these background data to make corresponding decisions. The page call instruction can be used to call certain sub-pages in the customer service program to obtain relevant information from these sub-pages, such as product details pages, etc. The search engine call instruction is used to call the search engine to assist customer service personnel in making corresponding decisions. Different from the target voice interaction instructions, these voice interaction instructions do not need to be replied to the target user.

[0047] In this embodiment, for any target question information, for the target voice interaction instruction in at least one voice interaction instruction initiated by the customer service staff for the target question information, at least part of the execution result information of the target voice interaction instruction can be selected as the target reply information of any target question information and displayed in the question and answer area. Among them, all the execution result information in the target voice interaction instruction can be selected as the target reply information of any target question information and displayed in the question and answer area, or part of the execution result information in the target voice interaction instruction can be selected as the target reply information of any target question information and displayed in the question and answer area, which can be determined specifically depending on the type of the target voice interaction instruction.

[0048] For example, in the case where the target voice interaction instruction is a speech-based instruction, the execution result information corresponding to the target voice interaction instruction may include multiple speech information, speech information 1 is "We are sorry for the inconvenience, please forgive me", speech information 2 is "I will help you check the logistics, please wait", and speech information 3 is "We will refund you as soon as possible, I hope you will wait patiently". The customer service staff can select a certain speech information from multiple speech information, such as speech information 1, as the target reply information of the target question information and display it in the question and answer area. For example, in the case where the target voice interaction instruction is a refund instruction, the execution result information corresponding to the target voice interaction instruction may be a refund card, and the AI ​​voice assistant can use the refund card as the target reply information of the target question information and display it in the question and answer area.

[0049] In this embodiment, an AI voice assistant associated with a customer service program is provided for customer service personnel. On the one hand, it is responsible for displaying the customer service interface provided by the customer service program; on the other hand, it can respond to and execute the voice interaction instructions of the customer service personnel and give the execution result information. For the voice instructions that need to be replied to by the user, it can select the content that can be used as the reply information from the execution result information and display it in the question and answer area of ​​the customer service interface to assist the customer service personnel in completing the interaction with the user. In this process, the customer service personnel can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of not being able to flexibly use peripherals for interaction due to the inconvenience of manual input, and thus solve the problem of low interaction efficiency caused by this. This solution is suitable for various interaction scenarios, especially those involving disabled people.

[0050] In an embodiment of the present application, the AI ​​voice assistant can serve as an assistant to the customer service personnel, execute various voice interaction instructions issued by the customer service personnel, and provide corresponding services to the customer service personnel; in an embodiment of the present application, the voice interaction instructions issued by the customer service personnel can be divided into different types of instructions according to the type of service provided by the AI ​​voice assistant to the customer service personnel, such as first-category instructions, second-category instructions, and third-category instructions.

[0051] Among them, the first type of instruction can enable the AI ​​voice assistant to execute the corresponding transaction in the data source associated with the customer service interface based on the target voice model. For example, the transaction can be a refund information push transaction and a historical price query transaction, etc. After executing the corresponding transaction, the transaction execution result can be displayed in the function area of ​​the customer service interface. For example, the transaction execution result corresponding to the refund information push transaction can be a refund card, and the transaction execution result corresponding to the historical price query transaction can be a price card. It is explained here that the AI ​​voice assistant can obtain the relevant information of the target user currently displayed on the customer service interface, and has access to and operation of the data source associated with the customer service interface. On this basis, the first type of instruction issued by the customer service staff can be executed to execute the corresponding transaction in the data source. In this embodiment, the relevant information of the target user includes but is not limited to: the identification information, account information, order information, etc. of the target user; the data source associated with the customer service interface includes but is not limited to: order information system, logistics information system, commodity information system and customer service knowledge base, etc.; the customer service knowledge base stores various problem information and its corresponding solutions, such as order anomalies and their solutions (such as canceling orders and initiating refund processes), logistics anomalies and their solutions (such as canceling orders and initiating refund processes), etc.

[0052] The first type of instructions can be understood as standard question instructions, that is, standard question instructions, such as order status query instructions, logistics information query instructions, historical price query instructions, evaluation information query instructions, promotion query instructions, refund information query instructions, and pre-sale size recommendation instructions, etc. The target voice model in the AI ​​voice assistant can be associated with a data source in the customer service interface. For example, the historical refund data in the data source can be associated with the refund control of the customer service interface, and the historical price data in the data source can be associated with the historical price viewing control of the customer service interface.

[0053] Among them, the second type of instruction refers to the voice instruction that the AI ​​voice assistant generates corresponding speech information based on the target voice model and displays the speech information on the interactive interface provided by the AI ​​voice assistant. In other words, the AI ​​voice assistant can respond to the second type of instruction and generate corresponding speech information, which is then displayed on the interactive interface provided by the AI ​​voice assistant for customer service personnel to select speech and reply to the target user. For example, the second type of instruction can be an instruction to generate soothing speech. The AI ​​voice assistant can respond to the second type of instruction and generate the following speech information using the target voice model: "Please select one of the following replies: 1. We are deeply sorry for this misunderstanding; 2. We are sorry for the inconvenience caused to you. Please forgive us." The second type of instruction can also be an instruction to end the speech. The AI ​​voice assistant can respond to the second type of instruction and generate the following speech information using the target voice model: "Please select one of the following replies: 1. Welcome to come again next time; 2. Thank you for your support. I hope to bring you a pleasant shopping experience."

[0054] Among them, the third category of instructions refers to standard instructions directly executed by the AI ​​voice assistant, such as AI voice assistant interface hiding instructions, session switching instructions, AI voice assistant shutdown instructions, swipe up instructions, swipe down instructions, favorites opening instructions, hosting instructions, click instructions, work instructions or off-get off work instructions, etc. Among them, the work instruction can be understood as the customer service program login instruction, and the off-get off work instruction can be understood as the customer service program exit instruction or the terminal shutdown instruction.

[0055] Further optionally, the voice interaction instructions of this embodiment may also include fourth and fifth category instructions. The fourth category instructions refer to instructions sent by the AI ​​voice assistant to the terminal device, such as assistant update instructions or assistant restart instructions, etc.; the fifth category instructions refer to instructions sent by the AI ​​voice assistant to the customer service program, such as customer service program restart instructions or customer service program update instructions.

[0056] In some optional embodiments, the AI ​​voice assistant may remain in a dormant state before engaging in voice interaction with the customer service staff. Before receiving at least one voice interaction instruction initiated by the customer service staff for any target question information, the AI ​​voice assistant may also be awakened based on any of the following implementations:

[0057] Implementation method 1: Based on the voice recognition module, respond to the wake-up command issued by the customer service staff in voice mode to wake up the AI ​​voice assistant. Specifically, the customer service staff can issue a wake-up command in voice mode, and the voice recognition module can perform voice recognition on the wake-up command to obtain the text information corresponding to the wake-up command, and compare the text information corresponding to the wake-up command with the preset wake-up text. If the match between the text information corresponding to the wake-up command and the preset wake-up text is greater than the preset match threshold, the AI ​​voice assistant can be woken up. The preset wake-up text refers to the text information required to wake up the AI ​​voice assistant, for example, it can be "Hello, special assistant", but is not limited to this.

[0058] Implementation method 2: Based on the head movement recognition module, respond to the wake-up command issued by the customer service staff through head movement to wake up the AI ​​voice assistant. Specifically, the head movement recognition module can call the camera of the terminal device to capture the facial image or video of the customer service staff, and perform head key point detection (such as eyes, nose tip, mouth corners, etc.) on the facial image or video, and then estimate the head posture according to the position of the detected head key points, such as pitch angle or deflection angle, etc. The head movement recognition module can determine the sight gaze position of the customer service staff under the current head posture according to the head posture of the customer service staff and the mapping relationship between the head posture and the sight gaze position. If the sight gaze position is on the icon of the AI ​​voice assistant, it can simulate the customer service staff clicking on the icon to wake up the AI ​​voice assistant. Alternatively, the head movement recognition module can also identify the type of head movement, such as nodding or shaking the head, according to the estimated head posture. If the type of head movement is the type of movement required to wake up the AI ​​voice assistant (such as nodding), the AI ​​voice assistant is woken up.

[0059] After waking up the AI ​​voice assistant in the above manner, the interactive interface provided by the AI ​​voice assistant can be displayed, and the text information corresponding to the wake-up command, such as "Hello, special assistant", can be displayed on the interactive interface. The interactive interface can be displayed in any position above the customer service interface in the form of a floating window or a drawer. This embodiment does not limit the display form and display position of the interactive interface. Figure 2 As shown, Figure 2 The example shows the interactive interface displayed in a floating window. Furthermore, after being awakened, the AI ​​voice assistant can actively ask the customer service staff what help they need, and the inquiry information can be output in the form of text display and voice broadcast at the same time, or in only one of the two ways, without limitation. Figure 2 The interactive interface shown in the figure displays the words "Hello, how can I help you?", which is the inquiry message that the AI ​​voice assistant proactively initiates to the customer service staff after being awakened.

[0060] Furthermore, during the operation of the AI ​​voice assistant, customer service personnel can hide the interactive interface at any time according to usage needs through hidden commands. Figure 3 and Figure 4 The state switching process of the interactive interface between the hidden state and the displayed state is described. Figure 3 As shown, before receiving the first type of instruction, if the interactive interface is in the display state, the interactive interface is hidden in response to the hidden instruction issued by the customer service staff through voice or head movement, and the icon of the AI ​​voice assistant is displayed on the customer service interface. This embodiment does not limit the display position of the AI ​​voice assistant icon on the customer service interface, and it can be displayed at any position on the customer service interface, such as the upper right corner or the lower left corner, etc. Or, Figure 4As shown in the figure, before receiving other types of instructions, if the interactive interface is in a hidden state, it will respond to the wake-up instruction issued by the customer service staff through voice or head movement, display the interactive interface, and display the text information corresponding to the wake-up instruction on the interactive interface. In this way, the customer service staff only needs to perform simple voice input or move their heads to complete the switching between the hidden state and the displayed state of the interactive interface.

[0061] In this way, customer service staff can be freed from manual input operations. They can quickly wake up the AI ​​voice assistant by voice or head movement, thereby improving interaction efficiency.

[0062] In some optional embodiments, the aforementioned step 13 "for any voice interaction instruction, executing any voice interaction instruction, and outputting the execution result information of any voice interaction instruction in a display and / or voice manner" can be implemented based on the following steps:

[0063] Step 131: Identify the instruction type of any voice interaction instruction according to the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction. The semantic information of the voice interaction instruction is used to describe the meaning, context and semantic relationship of the voice interaction instruction. The type marker word of the text instruction refers to: a word used to mark the type of the voice interaction instruction among multiple words obtained by word segmentation based on the text instruction. The type marker word refers to a marker word that appears in the voice interaction instruction to distinguish the type of the voice interaction instruction. Continuing from the first, second and third types of instructions divided above, the type marker words corresponding to the three types of instructions can be "question", "speech", "operation" in turn, or "A", "B", "C" and other marking information that can separate the three types of instructions. Taking "question", "speech" and "operation" as examples, the first type of instruction issued by the customer service staff includes "question" and specific instruction content; the second type of instruction issued by the customer service staff includes "speech" and specific instruction content; the third type of instruction issued by the customer service staff includes "operation" and specific instruction content. For example, if the customer service staff wants to query logistics information, they can issue "question: query logistics".

[0064] Among them, the instruction type of any voice interaction instruction can be identified according to the semantic information of any voice interaction instruction, and the instruction type of any voice interaction instruction can be identified according to the type marker words contained in any voice interaction instruction. The instruction type of any voice interaction instruction can also be identified according to the semantic information of any voice interaction instruction and the type marker words contained in any voice interaction instruction. This embodiment does not impose any restrictions.

[0065] Among them, different instruction types have different execution methods. For example, the execution method of some instruction types depends on the target language model, such as using the target language model to generate speech information; the execution method of some instruction types does not depend on the target voice model, such as calling the historical price information in the data source corresponding to the customer service interface and displaying it. For another example, the execution method of some instruction types does not depend on the data source corresponding to the customer service interface, such as using the target language model to generate emotional soothing information for customer service personnel; the execution method of some instruction types depends on the data source corresponding to the customer service interface, such as calling the historical refund information in the data source corresponding to the customer service interface and displaying it.

[0066] The result display methods of different instruction types are different. For example, if it is a first-type instruction, the result display method is: displayed in the function area of ​​the customer service interface, and if it is a second-type instruction or a third-type instruction, the result display method is: displayed on the interactive interface of the AI ​​voice assistant.

[0067] In the above step 131, when the instruction type of any voice interaction instruction is identified according to the semantic information of any voice interaction instruction, the text information conversion processing can be performed on any voice interaction instruction based on the voice recognition module, that is, the voice interaction instruction in audio form is converted into text form to obtain the text instruction corresponding to any voice interaction instruction. Based on the semantic information of the text instruction, the instruction type of any voice interaction instruction is identified. Among them, the following steps can be used to extract the semantic information of text instructions and then identify their instruction types: use a word segmenter to split the text instructions into multiple text units (such as words), and use part-of-speech to mark the grammatical role of each word (noun, verb, adjective, etc.); construct a syntactic tree or dependency relationship to understand the relationship between sentence components, such as subject, predicate, object, and modification relationship; perform entity recognition (Named Entity Recognition, NER) to extract specific information such as names, place names, institutions, time, and proper name entities; extract the relationship between entities, such as what someone did and when and where, so as to deeply understand the context; understand the text sentiment polarity (positive, negative, neutral); semantic role annotation to understand the meaning of words in the context of sentences, such as the metaphorical meaning of verbs; semantic analysis: use the target language model to deeply understand the context and associate the semantic relationship between words. After the semantic information of the text instructions is extracted through the above steps, the instruction type of any voice interaction instruction can be identified based on the semantic information of the text instructions.

[0068] In the above step 131, when identifying the instruction type of any voice interaction instruction based on the type marker words contained in any voice interaction instruction, any voice interaction instruction can be converted into text information based on the voice recognition module to obtain the text instruction corresponding to any voice interaction instruction; based on the type marker words contained in the text instruction, the instruction type of any voice interaction instruction is identified. Specifically, the text instruction can be split into multiple words using a word segmenter, and the instruction type of the voice interaction instruction can be determined based on the type marker words in the multiple words. For example, if the text instruction is split into multiple words containing the type marker word "refund", then the instruction type of the voice interaction instruction can be determined as the first category instruction based on the type marker word. In the above manner, the instruction type of any voice interaction instruction can be identified more accurately.

[0069] Step 132: according to the instruction type of any voice interaction instruction, in accordance with the execution mode adapted to the instruction type of any voice interaction instruction, execute any voice interaction instruction to obtain the execution result information of any voice interaction instruction. Specifically, the following situations may be included:

[0070] Case 1: When any voice interaction instruction is a first-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant so that the target voice model can execute the transaction corresponding to any voice interaction instruction in the data source associated with the customer service interface and output the transaction execution result as the execution result information of any voice interaction instruction. For example, when the voice interaction instruction is a logistics information query instruction, the target voice model can search for logistics information from the data source associated with the customer service interface according to the text instruction corresponding to the voice interaction instruction, and output the searched logistics information as the execution result information of the voice interaction instruction. For another example, when the voice interaction instruction is a refund information query instruction, the target voice model can search for refund information from the data source associated with the customer service interface according to the text instruction corresponding to the voice interaction instruction, and output the searched refund information as the execution result information of the voice interaction instruction.

[0071] Case 2: When any voice interaction instruction is a second-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant for speech requirement recognition and speech information generation processing, and at least one speech information corresponding to any voice interaction instruction is output as the execution result information of any voice interaction instruction. Among them, the target voice model can identify the speech requirement of the text instruction based on the semantic information of the text instruction corresponding to the voice interaction instruction, and determine the speech information corresponding to the speech requirement from the preset speech library as the execution result information according to the speech requirement.

[0072] Case 3: When any voice interaction instruction is a third-category instruction, the text instruction is matched in the standard instruction library supported by the AI ​​voice assistant. Specifically, the text instruction can be matched with the standard instruction stored in the standard instruction library. If there is a standard instruction in the standard instruction library whose similarity with the text instruction is greater than the preset similarity threshold, the standard instruction can be used as the target standard instruction. If there is no standard instruction in the standard instruction library whose similarity with the text instruction is greater than the preset similarity threshold, it means that the target standard instruction does not exist in the standard instruction library.

[0073] If the target standard class instruction is matched, the target standard class instruction can be executed to obtain the execution result information of any voice interaction instruction. If the target standard class instruction is not matched, the text instruction can be input into the target language model in the AI ​​voice assistant for instruction conversion processing. If the text instruction is converted into a standard class instruction, the converted standard class instruction is output. Specifically, the target language model can determine the standard class instruction that is closest to the semantic information from the standard class instructions stored in the standard class instruction library based on the semantic information of the text instruction.

[0074] If the text instruction cannot be converted into a standard instruction, that is, there is no standard instruction in the standard instruction library that is semantically close to the semantic information of the text instruction, in this case, prompt information can be output to the customer service personnel in the following ways: Method 1: voice output prompt information to remind the customer service personnel to re-enter any voice interaction instruction; Method 2: prompt information is displayed on the interactive interface to remind the customer service personnel to re-enter any voice interaction instruction; Method 3: voice output prompt information and prompt information is displayed on the interactive interface to remind the customer service personnel to re-enter any voice interaction instruction.

[0075] Step 133, according to the instruction type of any voice interaction instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant. Specifically, the result display method corresponding to the instruction type can be determined according to the instruction type and the correspondence between the instruction type and the result display method, so that the execution result information can be displayed in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant. In some optional embodiments, when any voice interaction instruction is a first-class instruction, the execution result information of any voice interaction instruction can be displayed in the functional area of ​​the customer service interface. Among them, the functional area is different from the question and answer area. When any voice interaction instruction is another type of instruction, the text information corresponding to any voice interaction instruction and its execution result information can be displayed on the interactive interface, and the interactive interface is located above the customer service interface.

[0076] In this way, voice interaction instructions can be executed and the execution result information can be displayed in a personalized manner according to different instruction types, in different execution methods and result display methods.

[0077] In some optional embodiments, the target voice interaction instruction may include at least one of the following voice instructions: a voice instruction belonging to the first category of instructions in at least one voice interaction instruction and containing a send control in the execution result information, and a voice instruction belonging to the second category of instructions in at least one voice interaction instruction. The send control is used to send at least part of the information in the execution result information to the question and answer area for display.

[0078] Among them, the AI ​​voice assistant can identify the target voice interaction instruction in the following way: for any voice interaction instruction in at least one voice interaction instruction, if any voice interaction instruction is a first-category instruction and the execution result information of any voice interaction instruction contains a sending control, any voice interaction instruction is used as the target voice interaction instruction; or, if any voice interaction instruction is a second-category instruction, any voice interaction instruction is used as the target voice interaction instruction.

[0079] Based on this, for the target voice interaction instruction in at least one voice interaction instruction, selecting at least part of the execution result information of the target voice interaction instruction as the target reply information of any target question information and displaying it in the question and answer area may include the following situations:

[0080] Case 4: When the target voice interaction instruction belongs to the first category of instructions and its execution result information includes a send control, the execution result information of the target voice interaction instruction is displayed in the function area of ​​the customer service interface, and the customer service staff can issue a send instruction by voice, or by triggering the send control by head movement, or by manually clicking the send control. In response to the send instruction issued by the customer service staff by voice or send control, part of the information associated with the send control in the execution result information of the target voice interaction instruction is sent directly from the function area to the question and answer area as the target reply information.

[0081] Figure 5 The example shows that the target voice interaction instruction belongs to the first type of instruction "refund information query instruction", such as Figure 5 As shown, in response to the sending instruction issued by the customer service staff by voice or sending control 1, the refund card associated with the sending control 1 in the execution result information of the target voice interaction instruction is sent directly from the function area to the question and answer area as the target reply information.

[0082] Case 5: When the target voice interaction instruction belongs to the second category of instructions, at least one speech information corresponding to the target voice interaction instruction is displayed on the interactive interface provided by the AI ​​voice assistant. Optionally, at least one speech information can be displayed directly on the interactive interface, or at least one speech information can be displayed dynamically on the interactive interface, or the voice input operation of the customer service staff can be simulated, corresponding to the microphone control in the interactive interface being pressed, which is not limited in this embodiment.

[0083] Respond to the selection instruction issued by the customer service staff in voice mode, select the target speech information, and display the target speech information in the input box corresponding to the question and answer area. Respond to the sending instruction issued by the customer service staff, and send the target speech information in the input box to the question and answer area. The customer service staff can issue the sending instruction by voice, or by triggering the send button in the input box by moving the head to issue the sending instruction, or by manually clicking the send button to issue the sending instruction, which is not limited in this embodiment. Figure 6 As shown, the customer service staff can select speech information M2 "I'm very sorry" as the target speech information from speech information M1-speech information M3, and issue a send instruction to send the target speech information in the input box to the question and answer area.

[0084] Case 6: For at least one non-target voice interaction instruction that belongs to the first category of instructions in the voice interaction instruction, the function area of ​​the customer service interface may display the execution result information of the non-target voice interaction instruction, such as Figure 7 As shown. Among them, non-target voice interaction instructions refer to voice instructions that do not require a reply to the target user. For example, since the historical price is only for the reference of the customer service staff and does not need to be sent to the user, the non-target voice interaction instruction of the first category of instructions can be a historical price query instruction. In addition, the non-target voice interaction instruction of the first category of instructions can also be a search engine result acquisition instruction, a customer service staff information acquisition instruction, and so on.

[0085] In response to the voice input operation initiated by the customer service staff in the input box of the question and answer area according to the execution result information of the non-target voice interaction instruction, the text information corresponding to the voice information input by the customer service staff is displayed in the input box. In other words, the customer service staff can refer to the execution result information of the non-target voice interaction instruction (such as search engine results or historical prices, etc.), and make decisions with the assistance of the execution result information and input the corresponding voice information.

[0086] In response to the sending command issued by the customer service staff, the text information in the input box is sent to the question and answer area as the target reply information of any target question information. Among them, the customer service staff can trigger the voice input control in the question and answer area by voice or head movement to initiate the voice input operation, which is not restricted in this implementation.

[0087] Through the above situations 4 to 6, customer service personnel can select at least part of the execution result information of the target voice interaction instruction as the target reply information of any target question information in an efficient and fast interactive manner and display it in the question and answer area.

[0088] In some optional embodiments, executing the target standard class instruction to obtain the execution result information of any voice interaction instruction may include the following situations:

[0089] Case 7: When the target standard instruction is a work instruction, at least one user session information that the customer service personnel is responsible for is displayed in the session area of ​​the customer service interface, such as Figure 8 Among them, any user session information may include any user's nickname and the message most recently sent by the user.

[0090] Case 8: When the target standard instruction is a session switching instruction, the user session information is switched in the question and answer area of ​​the customer service interface, and the user session information at least includes the target question information of the user currently switched to. Fig. 9 As shown, the dotted box represents the currently selected user session information, and the user session information is switched in the question and answer area of ​​the customer service interface, that is, the user session information 1 is switched to the user session information 2, thereby switching the target question information of user 1 displayed in the question and answer area to the target question information of user 2.

[0091] Based on the above situations 7 and 8, customer service personnel can control the customer service interface more quickly and efficiently.

[0092] It is to be noted that, in the process of customer service, the target user may be very dissatisfied and may speak ill of the customer service staff, which may cause emotional fluctuations in the customer service staff. Therefore, the AI ​​voice assistant provided in the embodiment of the present application can not only serve as an assistant to the customer service staff, but also as a good friend of the customer service staff to appease the customer service staff's emotions. The following will further explain:

[0093] When any target question information contains specific emotional words, emotion soothing information for customer service personnel can be generated based on the target language model. Among them, specific emotional words refer to sensitive words that affect the emotions of customer service personnel. The target language model can extract the semantic information of the target question information containing specific emotional words, and select the target emotion soothing sentence that matches the semantic information from the preset multiple emotion soothing sentences as the emotion soothing information. Afterwards, the emotion soothing information can be displayed on the interactive interface provided by the AI ​​voice assistant to soothe the customer service personnel. For example, when any target question information contains specific emotional words, the interactive interface provided by the AI ​​voice assistant can display the emotion soothing information of "Dear, the consumer is very excited now, please adjust your emotions, I will always be with you, come on" to soothe the customer service personnel.

[0094] It should be noted that, considering that the customer service staff may suddenly feel unwell during the service process, or have other reasons to temporarily leave their posts, the AI ​​voice assistant provided in the embodiment of the present application can not only serve as an assistant and a good friend of the customer service staff, but also as an emergency trustee of the customer service staff, acting as a customer service staff autonomously. The following will further explain:

[0095] When the customer service staff needs to leave their post due to physical discomfort or other reasons, they can issue an emergency hosting instruction by voice to use the AI ​​voice assistant to continue to complete the corresponding work tasks. Correspondingly, the AI ​​voice assistant can respond to the emergency hosting instruction and switch to the emergency working mode; in the emergency working mode, it can identify the target question information that has not been replied in the question and answer area, generate the target reply information of the target question information that has not been replied based on the target language model, and display the target reply information in the question and answer area. Among them, the target language model can extract the semantic information of the target question information that has not been replied, and select the target reply sentence that matches the semantic information from the preset multiple reply sentences as the target reply information. Among them, the target language model can be trained in advance based on a large number of training samples, so as to learn the contextual association relationship of the question information and the related reply rules, and then based on the learned contextual association relationship of the question information and the related reply rules, the target reply information of the target question information that has not been replied can be generated. Furthermore, the target language model can classify the target question information that has not been replied, so as to generate the target reply information according to the type of the target question information and the reply method corresponding to the type of the target question information. For example, for certain types of target question information, use a three-part response format, namely "summary answer-detailed answer-reference link"; for certain types of target question information, directly give a reference link or a summary answer.

[0096] The main functions of the AI ​​voice assistant provided in the embodiment of the present application are introduced above. The following takes the embedding of the AI ​​voice assistant in the customer service workbench (a customer service program implemented in the form of a website) as an example to describe the complete working process of the AI ​​voice assistant in detail:

[0097] The terminal device displays the homepage and login control of the customer service workbench. The customer service staff can use head movement to trigger the login control to log in, and the customer service interface can be displayed after logging in. The customer service staff can issue a wake-up command of "Hello, special assistant" by voice to wake up the AI ​​voice assistant to display the interactive interface. The customer service staff can issue a work command of "I want to go to work" by voice to display the customer service interface, and display at least one session information that the customer service staff needs to be responsible for in the conversation area of ​​the customer service interface, and display the current session information that needs to be responsible in the question and answer area. The session information includes at least one target question information consulted by the target user. On this basis, the customer service staff can issue various voice interaction commands to the AI ​​voice assistant, and the AI ​​voice assistant responds to various voice interaction commands issued by the customer service staff, and controls the content changes on the customer service interface based on the execution result information of the voice interaction commands.

[0098] For example, the customer service staff can issue a session switching command of "next session" by voice, and the AI ​​voice assistant can respond to the command to switch the current session to the next session to display at least one target question information consulted by the target user involved in the next session. Of course, the customer service staff can issue a session switching command of "previous session" by voice, and the AI ​​voice assistant can respond to the command to switch the current session to the previous session to display at least one target question information consulted by the target user involved in the previous session.

[0099] For example, the customer service staff can issue a command to generate soothing words to "comfort the user" through voice, and the AI ​​voice assistant can respond to the command and generate the following text message "Please choose one of the following replies: 1. We deeply apologize for this misunderstanding. 2. We are sorry for the inconvenience caused to you and please forgive us. 3. We will do our best to correct the mistake and apologize to you again", and give a voice prompt to the customer service staff "Please choose one of the following replies". The customer service staff can choose from multiple soothing words through voice, and the AI ​​voice assistant can respond to the customer service staff's selection operation and display the selected soothing words in the question and answer area of ​​the customer service interface, that is, send the selected soothing words to the target user.

[0100] For example, customer service staff can issue a hidden command of "hide" through voice, and the AI ​​voice assistant can respond to the command, hide the interactive interface, and display the icon of the AI ​​voice assistant on the customer service interface.

[0101] For example, customer service staff can issue a logistics information query command of "check logistics" by voice, and the AI ​​voice assistant can respond to the command and display the queried logistics information in the function area for reference by customer service staff. In response to the voice input operation initiated by customer service staff in the input box of the Q&A area based on the queried logistics information, the text information corresponding to the voice information input by the customer service staff is displayed in the input box: We have verified that there is an abnormality in your express logistics, and we will refund you immediately, please wait. In response to the send command issued by the customer service staff, the text information in the input box is sent to the Q&A area as the target reply information.

[0102] For example, customer service staff can issue a refund instruction of "help with refund" through voice, and the AI ​​voice assistant can respond to the instruction and display the refund card and the corresponding send control in the function area of ​​the customer service interface. Customer service staff can issue a send instruction through voice or by moving their head to trigger the send control. The AI ​​voice assistant can respond to the customer service staff's send instruction and send the refund card directly from the function area to the Q&A area.

[0103] Optionally, the AI ​​voice assistant can provide an AI expansion service, which can expand the text information corresponding to the voice message input by the customer service staff. For example, the text information corresponding to the voice message input by the user is: Don’t worry, the target text information obtained after expansion can be: Dear, don’t worry, I will help you check the previous communication records first.

[0104] Optionally, the AI ​​voice assistant can detect specific emotional words in the target question information consulted by the target user. When any target question information contains specific emotional words, it can generate emotion-soothing information for customer service personnel based on the target language model, such as "Your heart is full of endless energy. Face difficulties bravely, and you will find that your strength is far beyond your imagination", and display the emotion-soothing information on the interactive interface provided by the AI ​​voice assistant to soothe the emotions of the customer service personnel.

[0105] Optionally, when a customer service staff member suddenly feels unwell and needs to leave his post, he can issue an emergency custody instruction by moving his head or speaking, so that he can use the AI ​​voice assistant to continue to complete the corresponding work tasks.

[0106] In this way, AI voice assistants can more efficiently assist customer service personnel in completing interactions with target users. In this process, customer service personnel can complete interactions through simple voice input with the assistance of AI voice assistants, which can solve the problem of inability to flexibly use peripherals for interaction due to inconvenient manual input, and thus solve the problem of low interaction efficiency caused by this. This solution is suitable for various interaction scenarios, especially those involving people with disabilities.

[0107] In addition to the information interaction method in the aforementioned embodiment, the present application also provides an information interaction method, which is applied to an AI voice assistant on a terminal device, and the AI ​​voice assistant is associated with a target program on the terminal device. Fig.10 As shown, the method may include Fig.10 Steps shown:

[0108] Step 1001: display an interactive interface provided by a target program, where the interactive interface includes an interactive area, and the interactive area is used to display at least one interactive message initiated by a first user to a second user.

[0109] Step 1002: Based on the voice recognition module in the AI ​​voice assistant, receive at least one voice interaction instruction initiated by the second user for any interaction information.

[0110] Step 1003: execute any voice interaction instruction and output the execution result information of any voice interaction instruction in the form of display and / or voice.

[0111] Step 1004: for a target voice interaction instruction in at least one voice interaction instruction, select at least part of the execution result information of the target voice interaction instruction as target reply information of any interaction information and display it in the interaction area.

[0112] Optionally, the target program may be an instant messaging program, the interactive interface may be a chat window, the interactive area may be an information display area in the chat window, and the chat window may also include an information input area.

[0113] Optionally, the AI ​​voice assistant also includes a head movement recognition module; before receiving at least one voice interaction instruction initiated by the second user for any target question information, it also includes: based on the voice recognition module or the head movement recognition module, responding to a wake-up instruction issued by the second user through voice or head movement, waking up the AI ​​voice assistant; and displaying the interactive interface provided by the AI ​​voice assistant, and displaying text information corresponding to the wake-up instruction on the interactive interface.

[0114] Optionally, for any voice interaction instruction, execute any voice interaction instruction, and output the execution result information of any voice interaction instruction in display and / or voice form, including: identifying the instruction type of any voice interaction instruction based on the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, and different instruction types have different execution methods and result display methods; according to the instruction type of any voice interaction instruction, execute any voice interaction instruction in accordance with the execution method adapted to the instruction type of any voice interaction instruction to obtain the execution result information of any voice interaction instruction; and according to the instruction type of any voice interaction instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the interactive interface or on the interactive interface provided by the AI ​​voice assistant.

[0115] Optionally, the instruction type of any voice interaction instruction is identified based on the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, including: performing text information conversion processing on any voice interaction instruction based on a voice recognition module to obtain a text instruction corresponding to any voice interaction instruction; identifying the instruction type of any voice interaction instruction based on the semantic information of the text instruction and / or the type marker words contained in the text instruction.

[0116] Optionally, according to the instruction type of any voice interaction instruction, in accordance with an execution method adapted to the instruction type of any voice interaction instruction, execute any voice interaction instruction to obtain execution result information of any voice interaction instruction, including: when any voice interaction instruction is a first-category instruction, input the text instruction into the target language model in the AI ​​voice assistant, so that the target voice model executes the transaction corresponding to any voice interaction instruction in the data source associated with the interactive interface and outputs the transaction execution result as the execution result information of any voice interaction instruction; wherein the first-category instruction is a voice instruction in which the AI ​​voice assistant executes the corresponding transaction in the data source associated with the interactive interface based on the target language model and displays the transaction execution result in the functional area of ​​the interactive interface; when any voice interaction instruction is a second-category instruction In the case of a command, the text command is input into the target language model in the AI ​​voice assistant for speech requirement recognition and speech information generation processing, and at least one speech information corresponding to any voice interaction command is output as the execution result information of any voice interaction command; wherein, the second category of instructions refers to the voice command that the AI ​​voice assistant generates corresponding speech information based on the target voice model and displays the speech information on the interactive interface provided by the AI ​​voice assistant; when any voice interaction command is a third category of instructions, the text command is matched in the standard class instruction library supported by the AI ​​voice assistant. If the target standard class instruction is matched, the target standard class instruction is executed to obtain the execution result information of any voice interaction instruction; wherein, the third category of instructions refers to the standard class instructions directly executed by the AI ​​voice assistant.

[0117] Optionally, in the case where any voice interaction instruction is a third-category instruction, the method also includes: if the target standard-category instruction is not matched, inputting the text instruction into the target language model in the AI ​​voice assistant for instruction conversion processing; if the text instruction is converted into a standard-category instruction, outputting the converted standard-category instruction; if the text instruction cannot be converted into a standard-category instruction, outputting a prompt message by voice and / or displaying a prompt message on the interactive interface to remind the second user to re-enter any voice interaction instruction.

[0118] Optionally, according to the instruction type of any voice interaction instruction, the execution result information of any voice interaction instruction is displayed in the functional area of ​​the interactive interface or on the interactive interface provided by the AI ​​voice assistant, including: when any voice interaction instruction is a first-category instruction, the execution result information of any voice interaction instruction is displayed in the functional area of ​​the interactive interface, and the functional area is different from the question-and-answer area; when any voice interaction instruction is other types of instructions, the text information corresponding to any voice interaction instruction and its execution result information are displayed on the interactive interface, and the interactive interface is located above the interactive interface.

[0119] Optionally, it also includes: before receiving the first type of instruction, if the interaction interface is in a display state, responding to a hide instruction issued by the second user through voice or head movement, hiding the interaction interface, and displaying the icon of the AI ​​voice assistant on the interaction interface; or before receiving other types of instructions, if the interaction interface is in a hidden state, responding to a wake-up instruction issued by the second user through voice or head movement, displaying the interaction interface, and displaying the text information corresponding to the wake-up instruction on the interaction interface.

[0120] Optionally, the target voice interaction instruction includes a voice instruction belonging to the first category of instructions in at least one voice interaction instruction and the execution result information of which includes a send control, and / or a voice instruction belonging to the second category of instructions in at least one voice interaction instruction; for the target voice interaction instruction in at least one voice interaction instruction, at least part of the information in the execution result information of the target voice interaction instruction is selected as the target reply information of any target question information and displayed in the question and answer area, including: when the target voice interaction instruction belongs to the first category of instructions and its execution result information includes a send control, the execution result information of the target voice interaction instruction is displayed in the function area of ​​the interactive interface. In response to a sending instruction issued by the second user by voice or a sending control, part of the information associated with the sending control in the execution result information of the target voice interaction instruction is sent directly from the function area to the question-and-answer area as the target reply information; when the target voice interaction instruction belongs to the second category of instructions, at least one piece of speech information corresponding to the target voice interaction instruction is displayed on the interactive interface provided by the AI ​​voice assistant, and in response to a selection instruction issued by the second user by voice, the target speech information is selected and displayed in the input box corresponding to the question-and-answer area; and in response to a sending instruction issued by the second user, the target speech information in the input box is sent to the question-and-answer area.

[0121] Optionally, it also includes: for a non-target voice interaction instruction belonging to the first category of instructions in at least one voice interaction instruction, execution result information of the non-target voice interaction instruction is displayed in the function area of ​​the interactive interface; in response to a voice input operation initiated by a second user in the input box of the question and answer area based on the execution result information of the non-target voice interaction instruction, text information corresponding to the voice information input by the second user is displayed in the input box; and in response to a sending instruction issued by the second user, the text information in the input box is sent to the question and answer area as target reply information of any target question information.

[0122] Optionally, when the target standard type instruction is a go-to-work instruction, at least one user session information for which the second user is responsible is displayed in the conversation area of ​​the interactive interface; when the target standard type instruction is a conversation switching instruction, the user conversation information is switched in the question-and-answer area of ​​the interactive interface, and the user conversation information includes at least the target question information of the user currently switched to.

[0123] Optionally, it also includes at least one of the following operations: when any target question information contains specific emotional words, generating emotion soothing information for the second user based on the target language model, and displaying the emotion soothing information on the interactive interface provided by the AI ​​voice assistant; responding to an emergency hosting instruction issued by the second user in voice form, switching to an emergency working mode; in the emergency working mode, identifying target question information that has not yet been answered in the question and answer area, generating target reply information for the target question information that has not yet been answered based on the target language model, and displaying the target reply information in the question and answer area.

[0124] Optionally, the AI ​​voice assistant is embedded in the target program, or the AI ​​voice assistant and the target program are independent of each other, and an access portal of the target program is integrated into the AI ​​voice assistant, and the AI ​​voice assistant displays the interactive interface provided by the target program through the access portal.

[0125] The detailed implementation and beneficial effects of each step in the method of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0126] In this embodiment, an AI voice assistant associated with a customer service program is provided for customer service personnel. On the one hand, it is responsible for displaying the customer service interface provided by the customer service program; on the other hand, it can respond to and execute the voice interaction instructions of the customer service personnel and give the execution result information. For the voice instructions that need to be replied to by the user, it can select the content that can be used as the reply information from the execution result information and display it in the question and answer area of ​​the customer service interface to assist the customer service personnel in completing the interaction with the user. In this process, the customer service personnel can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of not being able to flexibly use peripherals for interaction due to the inconvenience of manual input, and thus solve the problem of low interaction efficiency caused by this. This solution is suitable for various interaction scenarios, especially those involving disabled people.

[0127] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 11 to 14 can be device A; for another example, the execution subject of steps 11 and 12 can be device A, and the execution subject of steps 13 and 14 can be device B; and so on.

[0128] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0129] Fig.11 FIG. 1 is a schematic diagram of a structure of an information interaction device provided by another exemplary embodiment of the present application. Fig.11 As shown, the device includes: a display module 1101, which is used to display a customer service interface provided by the customer service program, and the customer service interface includes a question and answer area, and the question and answer area is used to display at least one target question information for the target user to consult the customer service personnel; a receiving module 1102, which is used to receive at least one voice interaction instruction initiated by the customer service personnel for any target question information based on the voice recognition module in the AI ​​voice assistant; an execution module 1103, which is used to execute any voice interaction instruction for any voice interaction instruction, and output the execution result information of any voice interaction instruction in a display and / or voice manner; a selection module 1104, which is used to select at least part of the execution result information of the target voice interaction instruction for the target voice interaction instruction in the at least one voice interaction instruction as the target reply information of any target question information and display it in the question and answer area.

[0130] Optionally, the AI ​​voice assistant also includes a head movement recognition module; before receiving at least one voice interaction instruction initiated by the customer service personnel for any target question information, the receiving module 1102 is also used to: based on the voice recognition module or the head movement recognition module, respond to the wake-up instruction issued by the customer service personnel through voice or head movement to wake up the AI ​​voice assistant; and display the interactive interface provided by the AI ​​voice assistant, and display the text information corresponding to the wake-up instruction on the interactive interface.

[0131] Optionally, when the execution module 1103 executes any voice interaction instruction for any voice interaction instruction and outputs the execution result information of any voice interaction instruction in display and / or voice form, it is specifically used to: identify the instruction type of any voice interaction instruction according to the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, and different instruction types have different execution methods and result display methods; according to the instruction type of any voice interaction instruction, execute any voice interaction instruction in an execution method adapted to the instruction type of any voice interaction instruction to obtain the execution result information of any voice interaction instruction; and according to the instruction type of any voice interaction instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant.

[0132] Optionally, when the execution module 1103 identifies the instruction type of any voice interaction instruction based on the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, it is specifically used to: perform text information conversion processing on any voice interaction instruction based on the voice recognition module to obtain a text instruction corresponding to any voice interaction instruction; identify the instruction type of any voice interaction instruction based on the semantic information of the text instruction and / or the type marker words contained in the text instruction.

[0133] Optionally, when the execution module 1103 executes any voice interaction instruction according to the instruction type of any voice interaction instruction and in an execution mode adapted to the instruction type of any voice interaction instruction to obtain the execution result information of any voice interaction instruction, it is specifically used to: when any voice interaction instruction is a first-category instruction, input the text instruction into the target language model in the AI ​​voice assistant, so that the target voice model executes the transaction corresponding to any voice interaction instruction in the data source associated with the customer service interface and outputs the transaction execution result as the execution result information of any voice interaction instruction; wherein the first-category instruction is a voice instruction in which the AI ​​voice assistant executes the corresponding transaction in the data source associated with the customer service interface based on the target language model and displays the transaction execution result in the function area of ​​the customer service interface; in any voice When the interactive instruction is a second-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant for speech requirement recognition and speech information generation processing, and at least one speech information corresponding to any voice interactive instruction is output as the execution result information of any voice interactive instruction; wherein, the second-category instruction refers to a voice instruction in which the AI ​​voice assistant generates corresponding speech information based on the target voice model and displays the speech information on the interactive interface provided by the AI ​​voice assistant; when any voice interactive instruction is a third-category instruction, the text instruction is matched in the standard class instruction library supported by the AI ​​voice assistant. If the target standard class instruction is matched, the target standard class instruction is executed to obtain the execution result information of any voice interactive instruction; wherein, the third-category instruction refers to the standard class instruction directly executed by the AI ​​voice assistant.

[0134] Optionally, when any of the voice interaction instructions is a third-category instruction, the execution module 1103 is also used to: if the target standard-class instruction is not matched, input the text instruction into the target language model in the AI ​​voice assistant for instruction conversion processing; if the text instruction is converted into a standard-class instruction, output the converted standard-class instruction; if the text instruction cannot be converted into a standard-class instruction, output a prompt message by voice and / or display a prompt message on the interactive interface to remind the customer service staff to re-enter any of the voice interaction instructions.

[0135] Optionally, when the execution module 1103 displays the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant according to the instruction type of any voice interaction instruction, it is specifically used to: when any voice interaction instruction is a first-category instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface, and the functional area is different from the question-and-answer area; when any voice interaction instruction is another type of instruction, display the text information corresponding to any voice interaction instruction and its execution result information on the interactive interface, and the interactive interface is located above the customer service interface.

[0136] Optionally, the execution module 1103 is also used to: before receiving the first type of instruction, if the interactive interface is in a display state, respond to a hide instruction issued by the customer service personnel through voice or head movement, hide the interactive interface, and display the icon of the AI ​​voice assistant on the customer service interface; or before receiving other types of instructions, if the interactive interface is in a hidden state, respond to a wake-up instruction issued by the customer service personnel through voice or head movement, display the interactive interface, and display the text information corresponding to the wake-up instruction on the interactive interface.

[0137] Optionally, the target voice interaction instruction includes a voice instruction belonging to the first category of instructions among the at least one voice interaction instruction and whose execution result information includes a send control, and / or a voice instruction belonging to the second category of instructions among the at least one voice interaction instruction; when the selection module 1104 selects at least part of the information in the execution result information of the target voice interaction instruction as the target reply information of any target question information of the target question information and displays it in the question and answer area, it is specifically used for: when the target voice interaction instruction belongs to the first category of instructions and its execution result information includes a send control, the execution result of the target voice interaction instruction is displayed in the function area of ​​the customer service interface. In response to the sending instruction issued by the customer service staff by voice or the sending control, part of the information associated with the sending control in the execution result information of the target voice interaction instruction is sent directly from the function area to the question and answer area as the target reply information; when the target voice interaction instruction belongs to the second category of instructions, at least one speech information corresponding to the target voice interaction instruction is displayed on the interactive interface provided by the AI ​​voice assistant, and in response to the selection instruction issued by the customer service staff by voice, the target speech information is selected and displayed in the input box corresponding to the question and answer area; and in response to the sending instruction issued by the customer service staff, the target speech information in the input box is sent to the question and answer area.

[0138] Optionally, the selection module 1104 is also used for: for a non-target voice interaction instruction belonging to the first category of instructions in the at least one voice interaction instruction, displaying the execution result information of the non-target voice interaction instruction in the function area of ​​the customer service interface, responding to the voice input operation initiated by the customer service personnel in the input box of the question and answer area according to the execution result information of the non-target voice interaction instruction, displaying text information corresponding to the voice information input by the customer service personnel in the input box, and responding to a sending instruction issued by the customer service personnel, sending the text information in the input box as the target reply information of any target question information to the question and answer area.

[0139] Optionally, when the execution module 1103 executes the target standard class instruction to obtain the execution result information of any voice interaction instruction, it is specifically used to: when the target standard class instruction is a work instruction, display at least one user session information that the customer service personnel needs to be responsible for in the conversation area of ​​the customer service interface; when the target standard class instruction is a conversation switching instruction, switch the user conversation information in the question and answer area of ​​the customer service interface, and the user conversation information includes at least the target question information of the user currently switched to.

[0140] Optionally, the selection module 1104 is also used to: when any of the target question information contains specific emotional words, generate emotion soothing information for the customer service personnel based on the target language model, and display the emotion soothing information on the interactive interface provided by the AI ​​voice assistant; respond to the emergency hosting instruction issued by the customer service personnel in voice form, and switch to the emergency working mode; in the emergency working mode, identify the target question information that has not been answered in the question and answer area, generate target reply information for the target question information that has not been answered based on the target language model, and display the target reply information in the question and answer area.

[0141] Optionally, the AI ​​voice assistant is embedded in the customer service program, or the AI ​​voice assistant and the customer service program are independent of each other, and an access portal of the customer service program is integrated in the AI ​​voice assistant, and the AI ​​voice assistant displays the customer service interface provided by the customer service program through the access portal.

[0142] In this embodiment, an AI voice assistant associated with a customer service program is provided for customer service personnel. On the one hand, it is responsible for displaying the customer service interface provided by the customer service program; on the other hand, it can respond to and execute the voice interaction instructions of the customer service personnel and give the execution result information. For the voice instructions that need to be replied to by the user, it can select the content that can be used as the reply information from the execution result information and display it in the question and answer area of ​​the customer service interface to assist the customer service personnel in completing the interaction with the user. In this process, the customer service personnel can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of not being able to flexibly use peripherals for interaction due to the inconvenience of manual input, and thus solve the problem of low interaction efficiency caused by this. This solution is suitable for various interaction scenarios, especially those involving disabled people.

[0143] The above describes the internal functions and structure of the information interaction device. Fig.12 As shown, in practice, the information interaction device can be implemented as an electronic device, including: a memory 1201, a processor 1202 and a communication component 1203.

[0144] The memory 1201 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc.

[0145] Memory 1201 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0146] The processor 1202 is coupled to the memory 1201 and is used to execute the computer program in the memory 1201, so as to: display a customer service interface provided by the customer service program, the customer service interface including a question and answer area, and the question and answer area is used to display at least one target question information for the target user to consult the customer service personnel; based on the voice recognition module in the AI ​​voice assistant, receive at least one voice interaction instruction initiated by the customer service personnel for any target question information; for any voice interaction instruction, execute the any voice interaction instruction, and output the execution result information of the any voice interaction instruction in a display and / or voice manner; for the target voice interaction instruction in the at least one voice interaction instruction, select at least part of the execution result information of the target voice interaction instruction as the target reply information of any target question information and display it in the question and answer area.

[0147] Optionally, the AI ​​voice assistant also includes a head movement recognition module; before receiving at least one voice interaction instruction initiated by the customer service personnel for any target question information, the receiving module 1102 is also used to: based on the voice recognition module or the head movement recognition module, respond to the wake-up instruction issued by the customer service personnel through voice or head movement to wake up the AI ​​voice assistant; and display the interactive interface provided by the AI ​​voice assistant, and display the text information corresponding to the wake-up instruction on the interactive interface.

[0148] Optionally, when the processor 1202 executes any voice interaction instruction for any voice interaction instruction and outputs the execution result information of any voice interaction instruction in display and / or voice form, it is specifically used to: identify the instruction type of any voice interaction instruction according to the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, and different instruction types have different execution methods and result display methods; according to the instruction type of any voice interaction instruction, execute any voice interaction instruction in an execution method adapted to the instruction type of any voice interaction instruction to obtain the execution result information of any voice interaction instruction; and according to the instruction type of any voice interaction instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant.

[0149] Optionally, when the processor 1202 identifies the instruction type of any voice interaction instruction based on the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, it is specifically used to: perform text information conversion processing on any voice interaction instruction based on the voice recognition module to obtain a text instruction corresponding to any voice interaction instruction; identify the instruction type of any voice interaction instruction based on the semantic information of the text instruction and / or the type marker words contained in the text instruction.

[0150] Optionally, when the processor 1202 executes any voice interaction instruction according to the instruction type of any voice interaction instruction and in an execution mode adapted to the instruction type of any voice interaction instruction to obtain the execution result information of any voice interaction instruction, it is specifically used to: when any voice interaction instruction is a first-category instruction, input the text instruction into a target language model in the AI ​​voice assistant, so that the target voice model executes the transaction corresponding to any voice interaction instruction in the data source associated with the customer service interface and outputs the transaction execution result as the execution result information of any voice interaction instruction; wherein the first-category instruction is a voice instruction in which the AI ​​voice assistant executes the corresponding transaction in the data source associated with the customer service interface based on the target language model and displays the transaction execution result in the function area of ​​the customer service interface; in any voice interaction instruction When the interactive instruction is a second-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant for speech requirement recognition and speech information generation processing, and at least one speech information corresponding to any voice interaction instruction is output as the execution result information of any voice interaction instruction; wherein, the second-category instruction refers to the voice instruction in which the AI ​​voice assistant generates corresponding speech information based on the target voice model and displays the speech information on the interactive interface provided by the AI ​​voice assistant; when any voice interaction instruction is a third-category instruction, the text instruction is matched in the standard class instruction library supported by the AI ​​voice assistant. If the target standard class instruction is matched, the target standard class instruction is executed to obtain the execution result information of any voice interaction instruction; wherein, the third-category instruction refers to the standard class instruction directly executed by the AI ​​voice assistant.

[0151] Optionally, when any of the voice interaction instructions is a third-category instruction, the processor 1202 is also used to: if the target standard-class instruction is not matched, input the text instruction into the target language model in the AI ​​voice assistant for instruction conversion processing; if the text instruction is converted into a standard-class instruction, output the converted standard-class instruction; if the text instruction cannot be converted into a standard-class instruction, voice output prompt information and / or display prompt information on the interactive interface to remind the customer service personnel to re-enter any of the voice interaction instructions.

[0152] Optionally, when the processor 1202 displays the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant according to the instruction type of any voice interaction instruction, it is specifically used to: when any voice interaction instruction is a first-category instruction, display the execution result information of any voice interaction instruction in the functional area of ​​the customer service interface, and the functional area is different from the question-and-answer area; when any voice interaction instruction is another type of instruction, display the text information corresponding to any voice interaction instruction and its execution result information on the interactive interface, and the interactive interface is located above the customer service interface.

[0153] Optionally, processor 1202 is also used to: before receiving the first type of instruction, if the interactive interface is in a display state, respond to a hide instruction issued by the customer service personnel through voice or head movement, hide the interactive interface, and display the icon of the AI ​​voice assistant on the customer service interface; or before receiving other types of instructions, if the interactive interface is in a hidden state, respond to a wake-up instruction issued by the customer service personnel through voice or head movement, display the interactive interface, and display the text information corresponding to the wake-up instruction on the interactive interface.

[0154] Optionally, the target voice interaction instruction includes a voice instruction belonging to the first category of instructions among the at least one voice interaction instruction and the execution result information of which includes a send control, and / or a voice instruction belonging to the second category of instructions among the at least one voice interaction instruction; when the processor 1202 selects at least part of the information in the execution result information of the target voice interaction instruction as the target reply information of any target question information of the target question information and displays it in the question and answer area, it is specifically used for: when the target voice interaction instruction belongs to the first category of instructions and its execution result information includes a send control, the execution result of the target voice interaction instruction is displayed in the function area of ​​the customer service interface. In response to the sending instruction issued by the customer service staff by voice or the sending control, part of the information associated with the sending control in the execution result information of the target voice interaction instruction is sent directly from the function area to the question and answer area as the target reply information; when the target voice interaction instruction belongs to the second category of instructions, at least one speech information corresponding to the target voice interaction instruction is displayed on the interactive interface provided by the AI ​​voice assistant, and in response to the selection instruction issued by the customer service staff by voice, the target speech information is selected and displayed in the input box corresponding to the question and answer area; and in response to the sending instruction issued by the customer service staff, the target speech information in the input box is sent to the question and answer area.

[0155] Optionally, the processor 1202 is also used to: for a non-target voice interaction instruction belonging to the first category of instructions in the at least one voice interaction instruction, display the execution result information of the non-target voice interaction instruction in the function area of ​​the customer service interface; respond to the voice input operation initiated by the customer service personnel in the input box of the question and answer area according to the execution result information of the non-target voice interaction instruction, display text information corresponding to the voice information input by the customer service personnel in the input box; and respond to a sending instruction issued by the customer service personnel, send the text information in the input box as the target reply information of any target question information to the question and answer area.

[0156] Optionally, when the processor 1202 executes the target standard class instruction to obtain the execution result information of any voice interaction instruction, it is specifically used to: when the target standard class instruction is a work instruction, display at least one user session information that the customer service personnel needs to be responsible for in the conversation area of ​​the customer service interface; when the target standard class instruction is a conversation switching instruction, switch the user conversation information in the question and answer area of ​​the customer service interface, and the user conversation information includes at least the target question information of the user currently switched to.

[0157] Optionally, processor 1202 is also used to: in the case where any of the target question information contains specific emotional words, generate emotion soothing information for the customer service personnel based on the target language model, and display the emotion soothing information on the interactive interface provided by the AI ​​voice assistant; respond to the emergency hosting instruction issued by the customer service personnel in voice form, and switch to the emergency working mode; in the emergency working mode, identify the target question information that has not been answered in the question and answer area, generate target reply information for the target question information that has not been answered based on the target language model, and display the target reply information in the question and answer area.

[0158] Optionally, the AI ​​voice assistant is embedded in the customer service program, or the AI ​​voice assistant and the customer service program are independent of each other, and an access portal of the customer service program is integrated in the AI ​​voice assistant, and the AI ​​voice assistant displays the customer service interface provided by the customer service program through the access portal.

[0159] Further, if Fig.12 As shown, the electronic device also includes: a display 1204, a power component 1205, an audio component 1206 and other components. Fig.12 Only some components are shown schematically, which does not mean that the electronic device only includes Fig.12 In addition, Fig.12The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the working node. The working node of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server-side device such as a conventional server, a cloud server, or a server array. If the working node of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Fig.12 If the working node of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Fig.12 Components within the dashed box.

[0160] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0161] In this embodiment, an AI voice assistant associated with a customer service program is provided for customer service personnel. On the one hand, it is responsible for displaying the customer service interface provided by the customer service program; on the other hand, it can respond to and execute the voice interaction instructions of the customer service personnel and give the execution result information. For the voice instructions that need to be replied to by the user, it can select the content that can be used as the reply information from the execution result information and display it in the question and answer area of ​​the customer service interface to assist the customer service personnel in completing the interaction with the user. In this process, the customer service personnel can complete the interaction through simple voice input with the assistance of the AI ​​voice assistant, which can solve the problem of not being able to flexibly use peripherals for interaction due to the inconvenience of manual input, and thus solve the problem of low interaction efficiency caused by this. This solution is suitable for various interaction scenarios, especially those involving disabled people.

[0162] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0163] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0164] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0165] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0166] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0167] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0168] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0169] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0170] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0171] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0172] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0173] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0174] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0175] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An information interaction method, characterized in that: An AI voice assistant applied to a terminal device, wherein the AI ​​voice assistant is associated with a customer service program on the terminal device, and the method comprises: Displaying a customer service interface provided by the customer service program, wherein the customer service interface includes a question and answer area, wherein the question and answer area is used to display at least one target question information for a target user to consult a customer service staff; Based on the voice recognition module in the AI ​​voice assistant, receiving at least one voice interaction instruction initiated by the customer service staff for any target question information; For any voice interaction instruction, execute the any voice interaction instruction, and output the execution result information of the any voice interaction instruction in the form of display and / or voice; For the target voice interaction instruction in the at least one voice interaction instruction, at least part of the execution result information of the target voice interaction instruction is selected as the target reply information of any target question information and displayed in the question and answer area.

2. The method according to claim 1, characterized in that The AI ​​voice assistant also includes a head movement recognition module; before receiving at least one voice interaction instruction initiated by the customer service staff for any target question information, it also includes: Based on the voice recognition module or the head movement recognition module, responding to the wake-up command issued by the customer service staff through voice or head movement, waking up the AI ​​voice assistant; and Display the interactive interface provided by the AI ​​voice assistant, and display the text information corresponding to the wake-up instruction on the interactive interface.

3. The method according to claim 1, characterized in that For any voice interaction instruction, executing the any voice interaction instruction, and outputting the execution result information of the any voice interaction instruction in a display and / or voice manner, including: Identify the instruction type of any voice interaction instruction according to the semantic information of any voice interaction instruction and / or the type marker words contained in any voice interaction instruction, where different instruction types have different execution modes and result display modes; According to the instruction type of any voice interaction instruction, in accordance with the execution mode adapted to the instruction type of any voice interaction instruction, execute any voice interaction instruction to obtain execution result information of any voice interaction instruction; and According to the instruction type of any voice interaction instruction, the execution result information of any voice interaction instruction is displayed in the function area of ​​the customer service interface or on the interaction interface provided by the AI ​​voice assistant.

4. The method according to claim 3, characterized in that Identifying the instruction type of any voice interaction instruction according to the semantic information of any voice interaction instruction and / or the type marker word contained in any voice interaction instruction includes: Based on the speech recognition module, the speech interaction instruction is converted into text information to obtain a text instruction corresponding to the speech interaction instruction; Based on the semantic information of the text instruction and / or the type marker words contained in the text instruction, the instruction type of any voice interaction instruction is identified.

5. The method according to claim 4, characterized in that According to the instruction type of any voice interaction instruction, in accordance with the execution mode adapted to the instruction type of any voice interaction instruction, executing any voice interaction instruction to obtain execution result information of any voice interaction instruction, including: In the case where any of the voice interaction instructions is a first-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant, so that the target voice model executes the transaction corresponding to any of the voice interaction instructions in the data source associated with the customer service interface and outputs the transaction execution result as the execution result information of any of the voice interaction instructions; wherein the first-category instruction is a voice instruction for the AI ​​voice assistant to execute the corresponding transaction in the data source associated with the customer service interface based on the target language model and display the transaction execution result in the function area of ​​the customer service interface; In the case where any of the voice interaction instructions is a second-category instruction, the text instruction is input into the target language model in the AI ​​voice assistant to perform speech requirement recognition and speech information generation processing, and at least one piece of speech information corresponding to any of the voice interaction instructions is output as the execution result information of any of the voice interaction instructions; wherein the second-category instruction refers to a voice instruction in which the AI ​​voice assistant generates corresponding speech information based on the target voice model and displays the speech information on the interactive interface provided by the AI ​​voice assistant; In the case where any of the voice interaction instructions is a third-category instruction, the text instruction is matched in a standard class instruction library supported by the AI ​​voice assistant. If the target standard class instruction is matched, the target standard class instruction is executed to obtain the execution result information of any of the voice interaction instructions; wherein, the third-category instruction refers to the standard class instruction directly executed by the AI ​​voice assistant.

6. The method according to claim 5, characterized in that In the case where any of the voice interaction instructions is a third-category instruction, the method further includes: If the target standard class instruction is not matched, the text instruction is input into the target language model in the AI ​​voice assistant for instruction conversion processing. If the text instruction is converted into a standard class instruction, the converted standard class instruction is output; if the text instruction cannot be converted into a standard class instruction, a voice output prompt message is displayed and / or a prompt message is displayed on the interactive interface to remind the customer service staff to re-enter any of the voice interaction instructions.

7. The method according to claim 5, characterized in that According to the instruction type of any of the voice interaction instructions, the execution result information of any of the voice interaction instructions is displayed in the function area of ​​the customer service interface or on the interactive interface provided by the AI ​​voice assistant, including: In the case where any of the voice interaction instructions is a first-category instruction, the execution result information of any of the voice interaction instructions is displayed in a function area of ​​the customer service interface, and the function area is different from the question and answer area; In the case where any of the voice interaction instructions is other types of instructions, the text information corresponding to the any of the voice interaction instructions and the execution result information thereof are displayed on the interaction interface, and the interaction interface is located above the customer service interface.

8. The method according to claim 7, characterized in that Also includes: Before receiving the first type of instruction, if the interactive interface is in a display state, respond to a hiding instruction issued by the customer service staff through voice or head movement, hide the interactive interface, and display the icon of the AI ​​voice assistant on the customer service interface; or Before receiving other types of instructions, if the interactive interface is in a hidden state, the interactive interface is displayed in response to a wake-up instruction issued by the customer service staff through voice or head movement, and the text information corresponding to the wake-up instruction is displayed on the interactive interface.

9. The method according to claim 1, characterized in that: The target voice interaction instruction includes a voice instruction belonging to the first category of instructions among the at least one voice interaction instruction and including a voice instruction of a send control in the execution result information, and / or a voice instruction belonging to the second category of instructions among the at least one voice interaction instruction; For a target voice interaction instruction in the at least one voice interaction instruction, selecting at least part of the execution result information of the target voice interaction instruction as target reply information of any target question information and displaying it in the question and answer area includes: In the case where the target voice interaction instruction belongs to the first category of instructions and its execution result information includes a send control, the execution result information of the target voice interaction instruction is displayed in the function area of ​​the customer service interface, and in response to the send instruction issued by the customer service personnel by voice or the send control, part of the information associated with the send control in the execution result information of the target voice interaction instruction is directly sent from the function area to the question and answer area as the target reply information; In the case that the target voice interaction instruction belongs to the second category of instructions, at least one piece of speech information corresponding to the target voice interaction instruction is displayed on the interactive interface provided by the AI ​​voice assistant, and in response to the selection instruction issued by the customer service personnel in voice form, the target speech information is selected and displayed in the input box corresponding to the question and answer area; and in response to the sending instruction issued by the customer service personnel, the target speech information in the input box is sent to the question and answer area.

10. The method according to claim 9, characterized in that Also includes: For the non-target voice interaction instructions belonging to the first category of instructions among the at least one voice interaction instruction, the execution result information of the non-target voice interaction instruction is displayed in the function area of ​​the customer service interface, and in response to the voice input operation initiated by the customer service personnel in the input box of the question and answer area according to the execution result information of the non-target voice interaction instruction, text information corresponding to the voice information input by the customer service personnel is displayed in the input box, and in response to the sending instruction issued by the customer service personnel, the text information in the input box is sent to the question and answer area as the target reply information of any target question information.

11. The method according to any one of claims 1 to 10, characterized in that: Also includes at least one of the following operations: In the case where any of the target question information contains specific emotional words, generating emotional soothing information for the customer service personnel based on the target language model, and displaying the emotional soothing information on the interactive interface provided by the AI ​​voice assistant; In response to the emergency hosting instruction issued by the customer service staff in voice form, switch to the emergency working mode; in the emergency working mode, identify the target question information that has not been answered in the question and answer area, generate target reply information for the target question information that has not been answered based on the target language model, and display the target reply information in the question and answer area.

12. An information interaction method, characterized in that: An AI voice assistant applied to a terminal device, wherein the AI ​​voice assistant is associated with a target program on the terminal device, and the method comprises: Displaying an interactive interface provided by the target program, wherein the interactive interface includes an interactive area, and the interactive area is used to display at least one interactive message initiated by a first user to a second user; Based on the voice recognition module in the AI ​​voice assistant, receiving at least one voice interaction instruction initiated by the second user with respect to the interaction information; For any voice interaction instruction, execute the any voice interaction instruction, and output the execution result information of the any voice interaction instruction in the form of display and / or voice; For the target voice interaction instruction in the at least one voice interaction instruction, at least part of the execution result information of the target voice interaction instruction is selected as the target reply information of any interaction information and displayed in the interaction area.

13. The method according to claim 12, characterized in that The target program is an instant messaging program, the interactive interface is a chat window, the interactive area is an information display area in the chat window, and the chat window also includes an information input area.

14. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory to implement the steps in the method of any one of claims 1-11 and claims 12-13.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps of the method according to any one of claims 1 to 11 and claims 12 to 13.

16. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 11 and claims 12 to 13.

Citation Information

Patent Citations

  • Intelligent customer service matching method and device, terminal equipment and storage medium

    CN110427472A

  • Method for providing question consultation service and electronic equipment

    CN113724036A

  • Information interaction method, device and system

    CN114500419A

  • Customer service configuration, response and service method and device and storage medium

    CN114881690A

  • Voice service navigation method and device, equipment and storage medium

    CN116996619A

Cited By

  • Interaction method, electronic equipment, storage medium and program product

    CN121578972A