Intelligent voice interaction method and system, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides an intelligent voice interaction method and system, electronic equipment, a storage medium and a program product. The method comprises the steps that after a target user (such as a target merchant) suitable for outbound calling is determined and a corresponding outbound calling task is created according to a service task (such as an e-commerce operation service task), an intelligent dialogue robot constructed based on a preset model is called to execute the outbound calling task; the intelligent dialogue robot is used for carrying out voice interaction with the target user, and the dialogue text used for conveying the service information is generated by utilizing the preset model in the voice interaction process, so that the dialogue text is output to the user. The preset model comprises a language model, and the service information has a corresponding relation with the service task. It can be seen that the intelligent dialogue robot (also called the outbound robot) with high interaction ability and intelligence is created based on the language model and the like, and by means of the intelligent dialogue robot, the outbound cost can be reduced, and the information reaching rate can be increased.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent calling, in particular to an intelligent voice interaction method and system, an electronic device, a storage medium and a program product. BACKGROUND

[0002] In different scenarios, e-commerce platforms often need to actively contact merchants by phone to complete tasks such as message notification (such as violation reminder), operation guidance, question and answer, and activity invitation. At present, such telephone operations mainly rely on manual customer service for outbound calls. However, manual outbound calls have the problems of low efficiency and high cost. In order to improve the reach efficiency, some e-commerce platforms try to use automatic voice notification, SMS push or application message reminder to convey information. However, these traditional automatic methods can only realize one-way information transmission function and lack interaction ability with merchants, and cannot provide personalized response and problem solving according to the feedback of merchants, resulting in poor merchant experience, low conversion rate and task completion rate. Therefore, there is an urgent need for a flexible interaction and more intelligent reach method to replace manual outbound calls and realize automatic, efficient, low-cost and intelligent outbound calls to actively reach merchants. SUMMARY

[0003] The embodiments of the present application provide an intelligent voice interaction method, system, electronic device, storage medium and program product to reduce outbound call cost and improve reach rate.

[0004] In a first embodiment, the present application provides an intelligent voice interaction method, which comprises:

[0005] determining a service task;

[0006] determining a target user for outbound call and creating a corresponding outbound call task according to the service task;

[0007] calling an intelligent dialogue robot constructed based on a preset model to perform the outbound call task, to use the intelligent dialogue robot to interact with the target user by voice, and to generate dialogue text for conveying service information by using the preset model in the voice interaction process; wherein the preset model comprises a language model, and the service information has a corresponding relationship with the service task;

[0008] outputting the dialogue text to the target user.

[0009] In a second embodiment, the present application further provides an intelligent voice interaction method, which comprises:

[0010] determining a target merchant matched according to an e-commerce operation service task;

[0011] initiating a call to the target merchant by using the intelligent dialogue robot, so as to perform voice interaction with the target merchant by using the intelligent dialogue robot, and generating dialogue text for conveying e-commerce operation service information in the voice interaction process; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario;

[0012] outputting the dialogue text to the target merchant.

[0013] In a third embodiment, the present application further provides an intelligent voice interaction method, which comprises:

[0014] determining a matched target merchant according to an e-commerce operation service scenario;

[0015] initiating a call to the target merchant by using the intelligent dialogue robot, so as to perform voice interaction with the target merchant by using the intelligent dialogue robot, and streaming dialogue text for conveying e-commerce operation service information in the voice interaction process; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario;

[0016] streaming the dialogue text to the target merchant;

[0017] continuously listening to voice input of the target merchant in the streaming process;

[0018] if a trigger interrupt operation is listened to, the dialogue text is interrupted from being output, and the voice input is responded to by the intelligent dialogue robot.

[0019] In a fourth embodiment, the present application further provides an intelligent voice interaction method, which comprises:

[0020] determining a matched target merchant according to an e-commerce operation service scenario;

[0021] initiating a call to the target merchant by using the intelligent dialogue robot, so as to perform voice interaction with the target merchant by using the intelligent dialogue robot, and outputting dialogue text for conveying e-commerce operation service information in the voice interaction process; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario;

[0022] performing voice synthesis processing on the dialogue text to obtain dialogue voice with personification characteristics;

[0023] sending the dialogue voice to the target merchant.

[0024] In a sixth embodiment, the present application provides an intelligent voice interaction system, which comprises:

[0025] a server, which is deployed with an intelligent dialogue robot, the intelligent dialogue robot is constructed based on a preset model, and the preset model comprises a language model;

[0026] The server is configured to determine a service task, determine a target user matched with the service task and create a corresponding outbound task according to the service task, invoke the intelligent dialogue robot to execute the outbound task, use the intelligent dialogue robot to perform voice interaction with the target user, and output dialogue text for conveying service information by using the preset model in the voice interaction process, wherein the service information has a corresponding relationship with the service task.

[0027] The client is configured to output the dialogue text to the target user.

[0028] The service task includes an e-commerce operation service task, and the target user includes a target merchant.

[0029] In a sixth embodiment, the embodiments of the present application further provide an electronic device, including a memory and a processor. The memory is configured to store executable instructions. The processor is configured to implement the steps in the method embodiments provided by the present application by running the executable instructions.

[0030] In a seventh embodiment, the present application provides a computer-readable storage medium. The storage medium stores a computer program, and the computer program is executed by a computer to implement the steps in the method embodiments provided by the present application.

[0031] In an eighth embodiment, the present application provides a computer program product. The computer program product includes a computer program, and the computer program is executed by a processor to implement the steps in the method embodiments provided by the present application.

[0032] The scheme provided by the embodiments of the present application can be used to determine a target user (e.g., a target merchant) for an appropriate outbound call according to a service task (e.g., an e-commerce operation service task) and create a corresponding outbound call task, and then invoke an intelligent dialogue robot constructed based on a preset model to perform the outbound call task, so as to realize voice interaction between the intelligent dialogue robot and the target user and generate dialogue text for conveying service information by using the preset model in the voice interaction process. The preset model includes a language model. As can be seen, the present application relies on a language model and the like to create an intelligent dialogue robot (also referred to as an outbound call robot) with strong interaction capability and intelligence, and the use of the intelligent dialogue robot can reduce outbound call costs and improve information reach rate. Further, when the dialogue text is output to the target user (e.g., a target merchant), it is output in a streaming manner, and if it is determined that a break operation is triggered according to the voice input of the target user during the streaming output of the dialogue text, the output of the dialogue text will be interrupted, and the voice input of the target user will also be responded to by the intelligent dialogue robot. As can be seen, the present application realizes interruptible interaction, which greatly improves the real-time and naturalness of voice interaction between the intelligent dialogue robot and the user. The specific implementation of outputting the dialogue text includes performing voice synthesis processing on the dialogue text to obtain dialogue voice with anthropomorphic characteristics, and then sending the dialogue voice to the target user to realize broadcasting to the target user. The dialogue voice broadcasted by the present application has anthropomorphic characteristics, which can make the voice of the robot similar to that of a real person, and is conducive to improving the voice interaction experience and naturalness of the user. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are included to provide a further understanding of the present application, form a part of the present application and illustrate the illustrative embodiments of the present application and the explanation of the present application, and do not constitute improper limitations on the present application. In the drawings:

[0034] Figure 1 The technical architecture schematic diagram corresponding to the technical scheme provided by an exemplary embodiment of the present application;

[0035] Figure 2 The schematic diagram of an intelligent voice interaction system provided by an exemplary embodiment of the present application;

[0036] Figure 3 The flowchart schematic diagram of an intelligent voice interaction method provided by an exemplary embodiment of the present application;

[0037] Figure 4A The timing diagram of outbound operation provided by an exemplary embodiment of the present application;

[0038] Figure 4B The timing diagram of knowledge base creation provided by an exemplary embodiment of the present application;

[0039] Figure 5A A schematic diagram of online scheduling of outbound tasks is provided for an exemplary embodiment of the present application;

[0040] Figure 5B A schematic diagram of online AB experiment for outbound tasks is provided for an exemplary embodiment of the present application;

[0041] Figure 6 A timing diagram of voice interaction between an intelligent dialogue robot and a target user is provided for an exemplary embodiment of the present application;

[0042] Figure 7 A timing diagram of dialogue strategy automatic optimization link is provided for an exemplary embodiment of the present application;

[0043] Figure 8 A structural schematic diagram of an electronic device is provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0044] In the operation of an e-commerce platform (also known as an electronic commerce platform), for some scenarios (such as order delivery reminders, rule violation reminders, activity registration guidance, etc.), two main touch methods are relied on: outbound calls by human customer service and automated notifications. Outbound calls by human customer service are made by customer service personnel through telephone communication with merchants. This touch method has flexible interaction capabilities, but has problems such as high cost, low efficiency, and difficulty in unifying service quality. According to statistics, the daily outbound call volume of a single customer service personnel is only about 60, and the successful connection rate is about 6.7%, so the overall touch effect is not good. Automated notifications use methods such as voice notification, SMS push, or application message reminders to convey information, but have problems such as one-way information transmission, zero interaction capability, and poor scene adaptability. One-way information transmission means that only message notification can be completed, and multiple rounds of question guidance cannot be achieved. Zero interaction capability means that follow-up questions from merchants cannot be responded to in real time (such as order details inquiry, activity rule explanation, etc.), and poor scene adaptability means that the script template is fixed and cannot be dynamically adjusted according to the dialogue.

[0045] Therefore, it is necessary to provide a more intelligent touch method that can support flexible interaction, replace human outbound calls, and actively touch merchants in an automated, efficient, low-cost, and intelligent outbound call manner.

[0046] To solve the above problems, the application provides a solution, the basic idea is: based on language model (Large Language Model, LLM), automatic speech recognition (Automatic Speech Recognition, ASR) technology, text to speech synthesis (Text-to-Speech, TTS) technology, etc. Realize the generalization and full-duplex intelligent outbound touch platform, which can support the operation to train and configure the corresponding scene robot and touch task according to different scenes, and recycle and analyze the touch effect.

[0047] To make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0048] It should be noted that in the case of user information involved in the embodiments of the present application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal. In addition, the various models (including but not limited to language models or large models) involved in the present application are in line with relevant legal and standard regulations.

[0049] Before introducing each embodiment, some technical terms appearing in the present application are briefly introduced.

[0050] Outbound: refers to the process of one party (e-commerce operation customer service, but intelligent customer service) contacting another party (such as a merchant) through telephone or other voice communication methods to achieve a specific purpose (such as making the merchant participate in a promotion).

[0051] The intelligent outbound touch platform is an automated voice interaction system for enterprise-level users (such as e-commerce platforms). The platform integrates artificial intelligence models (AI models such as language models), automatic speech recognition (ASR) and text-to-speech synthesis (TTS) technologies to achieve efficient voice touch of target users (such as merchants). In addition, the platform supports intelligent interaction of multiple industries, multiple scenarios and multiple rounds of dialogue, has high configurability and expandability, can quickly build dedicated scene robots according to different needs, and recycle and analyze touch effect in real time.

[0052] Pre-set model, for the trained machine learning model. Among them, in this application, the pre-set model includes a language model (LLM). The language model (LLM) is an artificial intelligence model designed to understand and generate human language. Language models are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. The characteristic of LLM is large scale, containing billions of parameters, helping to learn complex patterns in language data. The embodiments of the present application do not limit the number of parameters supported by the pre-set model, aiming to meet the actual application needs.

[0053] Text-to-speech synthesis (TTS): is a technology that converts written text into natural speech, which is widely used in various scenarios such as voice assistants, navigation systems, accessibility tools, and educational software.

[0054] Automatic speech recognition (ASR): refers to the technology of converting human speech signals into text, which involves multiple processes, including audio signal acquisition, feature extraction, acoustic model, language model, and decoding. ASR technology is widely used in voice assistants, transcription services, telephone customer service systems, etc.

[0055] Voice activity detection (VAD): is a signal processing technology designed to automatically identify speech portions in audio signals and distinguish them from non-speech portions (such as noise or silence). This technology is very important for improving the quality of voice communication, reducing storage requirements, and optimizing the performance of speech recognition systems, etc. It mainly focuses on "whether there is someone speaking now"

[0056] SSE (Server-Sent Events): is an HTTP-based browser API for implementing one-way real-time communication from the server to the client.

[0057] Igraph knowledge base: is an intelligent knowledge management system based on large-scale graph structure, aiming to support semantic understanding, intent recognition, multi-round dialogue management and complex question and answer processing in enterprise-level applications. This knowledge base not only can store and manage structured and unstructured service knowledge, but also can establish the association relationship between entities through graph modeling, so as to improve the reasoning ability and interaction quality of intelligent systems.

[0058] Hologres (Holo for short): is a real-time analytical database that can seamlessly integrate with the big data ecosystem and provide fast OLAP (Online Analytical Processing) query capabilities. Hologres is particularly suitable for application scenarios that require instant querying and analysis of large-scale data.

[0059] KVCache (Key-Value Cache) is a key technology for storing and reusing intermediate calculation results in the language model (LLM) inference process. Specifically, KV Cache avoids repeated calculation by caching the Key / Value of the self-attention mechanism (Attention), realizes space-time trade-off, and is the core technology to solve the calculation redundancy of autoregressive generation.

[0060] To facilitate understanding of the technical solutions, the underlying technical architecture on which the embodiments of the present application are based is first briefly described. The technical solutions provided by the following embodiments are all implemented based on this technical architecture.

[0061] Figure 1 The technical architecture diagram of the technical solutions provided by the present application is shown, specifically, the technical architecture of the aforementioned intelligent outbound contact platform. As shown in Figure 1 The technical architecture includes: an outbound operation platform (specifically referred to as an intelligent outbound operation platform), online scheduling, an outbound communication dialogue system (including an outbound communication module, an intelligent dialogue link), an offline training and strategy optimization module. These parts are briefly introduced below.

[0062] I. Outbound operation platform

[0063] On the outbound operation platform, the operation processes involved include but are not limited to: scenario knowledge base creation, scenario instance creation, user (such as merchant) screening, outbound task creation and initiation, outbound result recovery, etc. Among them,

[0064] (11) Scene knowledge base creation, which is referred to as knowledge base creation hereinafter, is a structured and extensible knowledge set constructed for a specific service scene (such as a service scene involved in e-commerce platform operation (such as “618 promotion activities”, order delivery reminders, rule violation reminders, etc.)) to support intelligent voice dialogue in the service scene. The knowledge information stored in the knowledge base includes knowledge related to the corresponding service scene. The service scene refers to different operation services in different operation platforms. For example, in an e-commerce platform, a service scene refers to an e-commerce operation service scene involved in the e-commerce platform, such as various promotion activities (such as 618, Double 11, etc.) held by the platform, order delivery reminders, rule violation reminders, and other e-commerce operation service scenes. For another example, the service scenes involved in financial platform operation include but are not limited to: loan repayment reminders, customer rights activation guidance, credit card limit adjustment notifications, etc. Taking the example of creating a knowledge base for “618 promotion activities”, the knowledge base can include but is not limited to the following knowledge information: available product types, information on various preferential activities such as full-price reduction, time-limited discount, and gifts, coupon acquisition and use rules, registration process, order and logistics, after-sales service, etc. Using the knowledge information in the scene knowledge base, the intelligent dialogue robot can answer various inquiries from merchants based on the knowledge information stored in the knowledge base when contacting the merchants by phone in the future.

[0065] For details of the creation and updating of the knowledge base, please refer to the relevant description in other embodiments.

[0066] (12) The above-mentioned scene instance creation refers to creating a corresponding scene task instance for a service scene to better meet user needs. Creating a scene task instance includes but is not limited to setting goals, selecting relevant knowledge bases, configuring relevant dialogue strategies (and / or dialogue points), dialogue strategy conversion, dynamic data sources, etc. Among them,

[0067] The above-mentioned setting goals includes setting user (such as merchant) screening rules (also referred to as user screening rules) to achieve the above-mentioned user screening step. Of course, setting goals can also include other content, such as setting promotion channels (such as setting up promotion activities using social media for “618 promotion activities”, or releasing advance information to warm up activities in advance, etc.).

[0068] The above-mentioned selection of relevant knowledge bases refers to configuring associated knowledge bases for a service scene, for example, configuring a knowledge base created in advance for “618 promotion activities” for the service scene of “618 promotion activities”.

[0069] The configured dialogue strategy (dialogue points) mentioned above, also known as a communication strategy, refers to a series of rules and technical means used when a smart dialogue robot interacts with a user in a voice manner for a corresponding service scenario, to ensure the fluency, effectiveness and better user experience of the dialogue, which includes but is not limited to: opening speech design (for example, after a merchant answers an outbound call, how to introduce the "618 promotion activities" to the merchant to inquire about the merchant's participation intention), bottom-up words (for example, when the smart dialogue robot is in a flow limiting state, the set bottom-up words are used to respond to the merchant), information layered output (for example, first introduce the highlights of the activity, then explain the registration conditions, and finally provide registration guidance), friendly tone in the dialogue process, outbound time period (which is used to control the outbound behavior to be executed within a specific time range, such as executing outbound calls only between 9 am and 6 pm on weekdays), and the like.

[0070] The dialogue strategy conversion mentioned above refers to converting the dialogue strategy into a corresponding prompt, so that the outbound communication dialogue system (specifically, the smart dialogue link therein, more specifically, the smart dialogue robot) can be directly used.

[0071] The dynamic data source mentioned above can be dynamically adjusted according to the actual situation to maintain the accuracy and timeliness of the data. For example, the dialogue strategy can be dynamically adjusted and the knowledge base can be updated according to online outbound records.

[0072] It needs to be supplemented here that the scenario task instance created above is referred to as a service task in the method embodiments below.

[0073] (13) User screening refers to screening target users that meet all requirements according to set user screening rules. For example, the target user can be a merchant on an e-commerce platform.

[0074] (14) The creation and initiation of the outbound task can refer to: converting the scenario task instance (i.e., the service task) into a corresponding actual outbound task and performing outreach. Therefore, it can be understood that one outbound task corresponds to one created scenario task instance. When creating an outbound task, it includes defining the work order type, designing the work order template, defining the outreach strategy and the outreach approach. For specific details of the content here, please refer to the related content in the method embodiments below, which will not be described in detail here.

[0075] (15) Outbound call result collection, which refers to collecting various information generated in the outbound call process (which can also be referred to as online outbound call records, including outbound call results, voice chat information, etc.), for subsequent analysis and improvement. For example, for automatic tuning and optimization of online scheduling strategies for dialog strategies, outbound tasks, etc. to automatically optimize various preset models (such as language models (LLM), retrieval enhancement models (RAG)) built into intelligent dialog machines, etc. Among them, the voice chat information includes at least one round of voice chat messages generated and recorded during voice interaction between the user and the intelligent dialog robot. The voice chat messages include but are not limited to: the user's voice input, the user text obtained by performing voice recognition on the user's voice input, and the dialog text output by the intelligent dialog robot, etc.

[0076] II. Online scheduling

[0077] Online scheduling is used to schedule outbound tasks concurrently according to the configuration of the operation to achieve outbound automation. During the scheduling process, user fatigue, resource control (such as communication resources, model resources, etc. control), etc. are supported.

[0078] III. Outbound communication dialog system

[0079] The outbound communication dialog system is used to provide real-time full-duplex outbound services. In specific implementation, the outbound communication dialog system includes an outbound communication module and an intelligent dialog link.

[0080] The outbound communication module described above is a voice full-duplex module, which includes voice activity detection (VAD), automatic speech recognition (ASR), text-to-speech synthesis (TTS), silence detection, etc. components. For the functions of these VAD, ASR and TTS components, please refer to the relevant descriptions in other embodiments. Silence detection is the duration of silence after detecting that voice activity has stopped. Silence detection is usually performed after VAD has detected that voice activity has started and ended, and mainly focuses on "has the user finished speaking". For example, after VAD detects that voice activity has stopped, the silence detector starts timing, and if no voice is detected again within the set timeout, it is considered that the user has finished speaking (valid silence is detected); if voice is detected again within the timeout, the timer is reset, and it is considered that the user is still speaking. For silence detection, by setting a reasonable silence timeout, it can be avoided that the system mistakenly considers that the user has finished speaking when the user is only taking a short breath or thinking, and thus interrupts the user's speech too early.

[0081] The intelligent dialogue link is implemented based on a language model (LLM model) and the like, and includes an intelligent dialogue module. Understandably, the intelligent dialogue module is an intelligent dialogue robot constructed based on an LLM model, an RAG model, and the like. In a specific implementation, the intelligent dialogue link can be deployed in a TPP (Task-oriented Dialogue Platform) dialogue platform, which is a system specially designed to support the development and deployment of intelligent dialogue robots.

[0082] In this application, the intelligent dialogue robot is also referred to as an outbound robot, which can implement knowledge retrieval from a corresponding knowledge base by using an RAG model, thereby performing information enhancement (i.e., corresponding Figure 1 The prompt word (Prompt) dynamic expansion shown in the middle) on the input prompt based on the retrieved external knowledge, and then uses the enhanced input prompt as the input of the LLM model, so that the LLM model outputs the response content (referred to as dialogue text in this application) in a streaming manner. The inference acceleration techniques adopted by the LLM model in generating the response content include but are not limited to KV Cache (key-value cache) and quantization. Quantization may, for example, include converting parameters (such as weights and / or activations) that need to be quantized in the model from a high-precision representation (such as float16) to a low-precision representation (such as int8 / int4, etc.). The selected LLM model is compared and tested with different language models (such as Qwen2, Llama3, etc.) to evaluate the performance and effect of different language models in a specific service scenario, so as to select a suitable LLM model to generate the response content.

[0083] In addition, the intelligent dialogue link described above also includes an AB module for implementing various comparison tests (including comparison tests of different language models). Specifically, the test functions that can be implemented in the AB module include but are not limited to model AB, scene strategy AB, and engineering AB configuration. Model AB refers to comparison tests of different models (including different language models). Scene strategy AB refers to comparison tests of different strategies (including dialogue strategies). Engineering AB configuration refers to flexible adjustment of experimental parameters (such as experimental parameters configured for outbound task scheduling, such as target user groups, call duration, admission rules (including call time period, threshold configured for user fatigue (determined according to the frequency of outbound received by the user), etc.) through configuration management.

[0084] Exemplarily, in the e-commerce platform, the intelligent dialogue robot currently used for outbound call is based on a first language model, but considering that there is a more advanced second language model available, the performance and effect of the first language model and the second language model can be evaluated through an AB model (e.g., evaluating the difference between the two language models in response accuracy, dialogue response time, etc.). Assuming that the evaluation analysis shows that the second language model performs better than the first language model in most indicators, based on the implementation results, the LLM model in the intelligent dialogue robot can be upgraded to use the second language model instead of the first language model.

[0085] In addition to the above, the intelligent dialogue link can also include a scene knowledge retrieval module and a user data real-time query module. The scene knowledge retrieval module is used to load the knowledge information associated with the current dialogue scene (the service scene corresponding to the outbound call task) from the corresponding knowledge base to provide context-related knowledge support for the current dialogue and enhance the quality and accuracy of the response content (i.e., dialogue text) generated by the intelligent dialogue robot. Specifically, the composition of the scene knowledge retrieval module includes an Embedding service and a knowledge base (e.g., an Igraph knowledge base). The Embedding service can convert text into vector representation (i.e., embedding) to facilitate similarity calculation and knowledge retrieval based on similarity. The type of knowledge base can be a knowledge graph. The user data real-time query module refers to the ability to access and obtain relevant information of a user (e.g., a merchant). Taking a merchant as a user on the e-commerce platform as an example, the relevant information of the merchant includes but is not limited to: the merchant's promotion activity history participation, historical sales data, product price data, order fulfillment status, store rating changes, etc. Based on these relevant information of the merchant, the quality of the response content generated in the subsequent response content generation can be further improved. For example, in the scenario of guiding the merchant to register for the 618 activity, according to the registered merchant information, historical participation, and sold product categories, the dialogue text output by the intelligent dialogue robot can be "Hello, I am the platform operation little brother. I noticed that you have not registered for this year's 618 Midsummer Classic. This activity will bring billion-level exposure resources, and we have reserved a special recruitment channel for you. Do you need me to introduce the participation method in detail?".

[0086] IV. Offline training and strategy optimization module

[0087] The offline training and strategy optimization module is used to train the model using historical data to improve the intelligence level of outbound call and achieve automatic optimization of dialogue strategy (also known as communication strategy) through reinforcement learning.

[0088] It needs to be supplemented here that the automatic tuning on the dialogue strategy (also known as the communication strategy) realized through reinforcement learning includes reward model training combined with service goals to realize self-tuning of the dialogue strategy. Understandably, the reward model is trained based on specific service goals to guide the optimization of the dialogue strategy.

[0089] The historical data includes online outbound records that have been accumulated, and the online outbound records include artificial outbound data, which can be used for targeted model fine-tuning and alignment to realize a low-latency, high-personalized language model-based intelligent dialogue robot in an outbound scenario. The "model alignment" function here includes ensuring consistency and coordination between models so that they can work together to achieve the expected service goals. This consistency not only involves the consistency of model output results, but also includes the coordination and unification of model training, updating, parameter setting, etc. In addition, the online outbound records also include outbound data generated during the recorded intelligent outbound process. Through this data backflow method, model parameters can be optimized, and the dialogue quality of the intelligent dialogue robot can be improved.

[0090] The online outbound records are obtained by result recovery of online outbound, and result recovery refers to collecting and recording all information and data obtained through online outbound, such as but not limited to voice interaction content (including text records, related voice files, etc.), the status of the called number (such as connected, not connected, busy, etc.), user feedback information (such as user needs, opinions or problems), etc. The online outbound records obtained through these recoveries can be used for subsequent analysis and improvement (such as optimizing the model and adjusting the dialogue strategy).

[0091] For example, by processing and analyzing the online outbound records (including ETL processing of the online outbound records (including extraction, transformation, and loading)), user intent can be obtained to understand user needs and behavior, and different processing paths or communication channels can be used for user intent. In addition, through analysis and processing of online outbound records, reason clustering analysis can be performed on outbound failure or inefficient dialogue to obtain reason clustering results, such as reason clustering results including: inappropriate time, repeated disturbance, content not understood, etc. According to the reason clustering results, strategy self-adjustment (automatic adjustment of dialogue strategy) can be performed, such as but not limited to: adjusting the outbound time, switching communication channels (such as switching to SMS notification if the user is not connected multiple times), changing the opening words, etc.

[0092] The technical scheme corresponding to the technical architecture mentioned above can be deployed on a server, which can be a server, a server cluster, a virtual server, a cloud, or the like, and the embodiment does not make a specific limitation on this. The server can provide relevant functional services for the client. For example, the server can make an outbound call to the user (such as a merchant) on the client side. Specifically, the server can make an outbound call to the user (such as a merchant) on the client side to perform voice interaction, so as to complete information notification, status confirmation, activity promotion, and the like. The client device can be, but is not limited to, a smart phone, a smart wearable device, a tablet computer, a notebook computer, a desktop computer, and the like.

[0093] Therefore, Figure 2 It is also shown that the intelligent voice interaction system provided by an embodiment of the application includes a server 100 and a client 200. Wherein,

[0094] The server 100 is deployed with an intelligent dialogue robot, and the intelligent dialogue robot is constructed based on a preset model, and the preset model includes a language model.

[0095] In an example, the server is configured to determine a service task (a created task instance created for a certain service scenario), determine a target user matched with the service task and create a corresponding outbound call task according to the service task, and invoke the intelligent dialogue robot to execute the outbound call task to perform voice interaction with the target user using the intelligent dialogue robot and output dialogue text for conveying service information in the voice interaction process using the preset model. The service information has a corresponding relationship with the service task.

[0096] In addition, the client is configured to output the dialogue text to the target user.

[0097] In the above, the service task includes an e-commerce operation service task, and the target user includes a target merchant.

[0098] It should be noted that the technical scheme of the application is mainly introduced by taking the outbound call scenario of the e-commerce platform as an example. In fact, the technical architecture provided by the application can also be applied to other specific outbound call scenarios, such as the outbound call scenario of a financial platform.

[0099] The technical scheme provided by the application will be described below in the form of a method embodiment.

[0100] Figure 3 A flowchart of an intelligent voice interaction method provided by an embodiment of the application is shown. The execution subject of the method provided by the embodiment can be the server in the system. As shown in the figure, Figure 3 The method can include the following steps:

[0101] 101, determine a service task;

[0102] 102, determine a target user to be called according to the service task and create a corresponding call task;

[0103] 103, call an intelligent dialogue robot constructed based on a preset model to perform the call task, use the intelligent dialogue robot to interact with the target user, and generate dialogue text for conveying service information by using the preset model during the voice interaction; wherein the preset model includes a language model, and the service information has a corresponding relationship with the service task;

[0104] 104, output the dialogue text to the target user.

[0105] In 101 above, it can be understood that the service task is a specific scene task instance created for a certain service scenario. Therefore, in this application, the creation of the service task is also referred to as the creation of the scene task instance.

[0106] Figure 4A The creation of the scene task process shown in the above Figure 4A , accordingly, creating a service task (i.e. a scene task) in this application can include setting a corresponding target for the service task, selecting a suitable knowledge base, and providing a corresponding dialogue strategy. Among them, Figure 4A The dialogue strategy conversion shown in the above can refer to the conversion of the dialogue strategy into a prompt word (Prompt), i.e. converting the dialogue strategy into a system (such as an intelligent dialogue robot) understandable prompt word (Prompt).

[0107] For details of setting a corresponding target, configuring a related knowledge base, and a dialogue strategy, please refer to the related content described in the foregoing Figure 1 description, which will not be described in detail here.

[0108] Among them, the service task refers to different things on different operation platforms. For example, in the e-commerce operation platform, the service task can be but not limited to a certain type of promotion activity (such as 618 activity) reminder, order delivery reminder, and violation reminder.

[0109] The target set for the service task is called a call target, and according to this call target, user screening can be performed to screen out target users suitable for calling.

[0110] Therefore, in a specific implementable technical solution, the above-mentioned 102 "determining a target user to be called according to the service task" can include:

[0111] 1021, obtain a call target set for the service task, the call target including user screening rules;

[0112] 1022. Filter out target users who need outbound calls according to the user screening rule.

[0113] Among them, in the e-commerce platform operation scenario, the target users who need to be called outbound include merchants.

[0114] For example, assume that the service task is "618 promotion reminder", and the corresponding user screening rule is: initiate a reminder call to merchants who have not signed up for the 618 promotion. Then, see Figure 4A As shown, the outbound call operation platform can send an information request to the user information center. This request carries a merchant identity identifier (such as nick (merchant nickname or store nickname) / sellerId (merchant unique ID)). The user information center responds to the request and queries the corresponding merchant information (such as name, contact information, store name, business status, event participation, etc.) according to the merchant identity identifier set in the request, and returns the queried merchant information to the outbound call operation platform, so that the outbound call operation platform can screen out target merchants who have not registered for the 618 promotion activities based on the merchant information.

[0115] Furthermore, in the above 102, creating a corresponding outbound call task may refer to: converting a service task into an actual outbound call task. Figure 1 As shown, it may include: defining corresponding work order types for outbound call tasks, setting standardized work order templates, defining contact strategies and contact channels. Work order types include but are not limited to reminders, complaints, etc. Defining corresponding work order types for outbound call tasks can facilitate subsequent classification management and statistical analysis of outbound calls. The design of work order templates can be used to determine the standardization and completeness of information incentives. In addition, the defined contact strategies include outbound call task priority, call time window, call retry mechanism (such as re-calling after 30 minutes after a call failure, no more calls after the number of call failures reaches a set number, and reasons for call failures such as not connected, busy, and power off), etc., which can determine the effective execution of outbound call tasks. Contact channel refers to determining which channel to use to reach the user, such as telephone.

[0116] For example, assuming that the service task corresponding to the outbound call task is "618 promotional event reminder", the work order type of the outbound call task can be defined as "reminder work order", and the format of the work order template involved can record user feedback and event registration status, etc. The formulated contact strategy includes giving priority to calling high-value merchants, avoiding rest time, etc., and the contact method is to contact the merchants by phone.

[0117] The outbound task is scheduled and initiated online to perform the outbound task by the intelligent dialogue robot, so as to realize voice interaction with the corresponding target user by the intelligent dialogue robot. The outbound task can be scheduled concurrently, and the user fatigue, communication resources, model resources and the like of the outbound task are controlled during the concurrent scheduling. Based on this, in addition to the step 103, the method provided by the application can further include the following steps:

[0118] S11, performing management and control processing on the outbound task;

[0119] S12, triggering the execution of the outbound task based on the intelligent dialogue robot after the management and control pass;

[0120] The management and control pass includes at least one of the following: the outbound task meets the admission rule, the communication resource configuration of the outbound task is completed, and the workload of the intelligent dialogue robot is less than a preset load threshold. The admission rule includes that the fatigue of the target user is less than a preset fatigue threshold, and the fatigue is determined according to the frequency of the outbound call received by the target user.

[0121] In the above, the outbound call frequency refers to the number of calls received per unit time (such as 24 hours or a week, etc.), and the higher the outbound call frequency received by the target user, the higher the fatigue. In addition to determining the fatigue according to the outbound call frequency received by the target user, the fatigue can also be comprehensively evaluated in combination with other multi-dimensional information of the target user, such as the time interval from the last outbound call received by the target user to the current time, the number of consecutive unattended outbound calls or outbound calls hung up, user behavior feedback of the target user, the current environment of the user, etc. The user behavior feedback includes: historical interaction attitude (such as whether to explicitly express "no more calls" or other refusal statements, whether to frequently end the conversation, etc.), call completion rate (whether to listen to the conversation content completely or hang up in the middle, etc.), and the like. The lower the call completion rate, the higher the fatigue. The current environment of the user includes: the current time (such as calling during non-working hours (such as late at night, lunch break), holidays, etc., which is easy to cause user dissatisfaction), the current state of the user (such as whether the user is driving, etc. Unsuitable state for answering).

[0122] In addition, the communication resource configuration of the outbound task includes but is not limited to: network bandwidth, number resource (such as virtual number), etc. The workload of the intelligent dialogue robot includes: the number of dialogue tasks being processed in a time period and the degree of occupation of system resources (CPU, memory, network, etc.) by the intelligent dialogue robot. When the workload of the intelligent dialogue robot exceeds the preset load threshold, the intelligent dialogue robot is controlled to limit the flow to prevent the intelligent dialogue robot from performance degradation, response delay and even crash due to too many processing requests.

[0123] Figure 5AA flowchart of initiating online scheduling of outbound tasks is shown. In the entire online scheduling process, outbound task scheduling is achieved through flexible configuration and concurrent execution of outbound tasks, and fine-grained control of user (such as merchant) fatigue, resource (such as communication resource, model resource), etc. is supported during the process. Among them, the model resource control can be understood as referring to: according to the workload of the intelligent dialogue robot, the intelligent dialogue robot is controlled in a flow limiting manner. Specifically, as shown in Figure 5A The outbound operation platform is the starting point of the entire outbound task scheduling process, which includes multiple outbound tasks (such as outbound task 1, outbound task 2, outbound task 3, etc.). These outbound tasks will be submitted to the "task delivery portal", where the outbound tasks will be preliminarily processed and distributed. Specifically, after the outbound task passes through the task delivery portal, it will first enter the "admission rule" link, where the outbound task will be screened and verified according to the pre-set admission rule, to ensure that only outbound tasks that meet the conditions can continue to be executed. If the outbound task does not meet the conditions, it will fail quickly and record the failure reason to return to the outbound operation platform. For example, in the "admission rule" link, the fatigue of the target user corresponding to the outbound task will be controlled. Only when the fatigue of the target user is less than the pre-set fatigue threshold, the outbound task will enter the next link for continuous execution. Through fatigue control in the "admission rule" link, the present application can avoid frequent disturbance of the user (such as the merchant), and ensure that the user will not feel tired or dissatisfied due to too many outbound calls. After the "admission rule" link, the outbound task will be forwarded to the communication merchant. The communication merchant may be a cloud communication service or a corresponding intelligent customer service system (such as a small honey intelligent customer service system in an e-commerce platform). In this communication merchant link, communication resource control (such as reasonable allocation of network bandwidth, virtual number scheduling) will be performed on the outbound task to ensure that the line will not be congested or the service quality will not be reduced due to a large number of outbound tasks. After the outbound task is processed by the communication merchant, it will enter the "flow limiting" link, which can prevent a large number of outbound tasks from flowing in within a certain period of time, causing system overload or crash. In the "flow limiting" link, the model resource will be controlled to ensure that the model resource used to process the outbound task can be reasonably allocated and used. Then, it will enter the TPP dialogue link, where the intelligent dialogue robot will start voice interaction with the target user.

[0124] Figure 5B The application architecture of the AB module used in the experiment in the outbound operation platform is shown. In this architecture, through experiment management, experiment group management, AB module, outbound scheduling module, integrated communication system (such as Figure 1The A / B test of the outbound call task is implemented. Specifically, the A / B test experiment design process of the outbound call task includes an experiment design phase and an experiment execution phase. The experiment design phase includes: in the online experiment platform, creating a corresponding experiment through an experiment management module, and defining multiple experiment groups for the created experiment in an experiment group management module. Each experiment group is assigned a unique experiment ID and is bound to a corresponding outbound call task. The experiment execution phase includes: according to the experiment ID bound to the outbound call task, calling the corresponding experiment group information through the AB module to obtain the corresponding experiment parameters; further, the outbound call scheduling module initiates scheduling for the outbound call task according to the experiment parameters, and assigns the outbound call task to the integrated communication system, so as to execute the outbound call operation through the integrated communication system at the time, and return the call Session data (including call ID, call state (such as completed, not connected, hung up in the middle, etc.), communication content, user feedback, etc.), so that the outbound call operation platform can analyze the experimental results according to the returned call Session data, evaluate the performance of different experiment groups, and thus optimize the corresponding dialogue strategy of the outbound call task.

[0125] For example, for the outbound call task of the 618 promotion activity, dialogue strategy A is initially used when executing this outbound call task. Then, as time goes by, dialogue strategy B is obtained by optimizing the dialogue strategy according to the historical call Session data corresponding to this outbound call task. For dialogue strategy B and dialogue strategy A, combined with the historical call Session data of the outbound call task, the performance of the two dialogue strategies on the merchant is evaluated through the A / B test experiment. Figure 5B The content is shown, an A / B test experiment can be designed to evaluate the effect of the two different dialogue strategies on the merchant. If dialogue strategy B is better than dialogue strategy A, then dialogue strategy B will be used in subsequent execution of the outbound call task.

[0126] After entering the intelligent dialogue robot and the target user for voice interaction through the above steps S11-S12, the user voice of the target user is collected in real time during the voice interaction process, and the merchant is replied according to whether the intelligent dialogue robot is in the flow limiting state.

[0127] Figure 6 An exemplary flowchart of voice interaction with the target user through the intelligent dialogue robot is shown. As shown in Figure 6As shown, assuming the target user is a merchant, after the merchant answers the call, the outbound communication system detects the merchant's speech, converts the detected speech into text through ASR, and performs an LLM request (requests a language model). The TTP responds to this request, and if it determines that its built-in language model is in a throttling state, it will use the corresponding fallback script to reply to the merchant's speech and hang up the call. If it determines that its built-in language model is not in a throttling state, it will initialize the interaction context (also known as the session context) according to the corresponding session identifier (session_id) of the voice interaction, ensure the coherence of the conversation during the voice interaction process, and perform query enhancement if it is determined that the RAG condition is met. Query enhancement refers to a series of processes such as expansion, rewriting, completion, or voice deepening of the original query input by the user to improve the expression ability of the query. For example, according to the above determination that the merchant is asking "How to sign up for this promotion activity?", the merchant's inquiry will be rewritten as "What is the specific preferential sign-up process for the 618 promotion activity" after query enhancement. Then, the enhanced inquiry is converted into a vector representation (embedding), and relevant knowledge is retrieved from the corresponding knowledge base based on the vector representation, so as to perform dynamic knowledge filling according to the retrieval result, and generate a reply text, which is the dialogue text for the merchant's inquiry. If the response time for the merchant's inquiry is too long (e.g., more than 600 milliseconds), the merchant may feel that the waiting time is too long, which affects the merchant's experience. To avoid this situation, the application triggers the "placeholder reply" mechanism to use some short and general reply sentences as "placeholders" and plays these placeholders to the merchant to inform the merchant that the system is processing and there is no lag or no response. For example, when the "placeholder reply" mechanism is triggered, a short reply such as "OK, I'm checking how to sign up for the 618 promotion activity..." or "Please wait a moment, I'll answer your question soon" can be played to the merchant. During the placeholder reply process, the background can continue to complete data processing and answer generation, so as to play the generated complete and formal reply, for example: "You can sign up for the 618 full-reduction activity by following these steps: first, log in to the merchant's back office; second, go to 'Marketing Center'; third, select 'Full-Reduction Activity' and submit an application……"

[0128] Based on the above description Figure 6 During the voice interaction process between the intelligent dialogue robot and the target user in step 103, the following steps can also be included:

[0129] S21, during the voice interaction process, collecting the user speech of the target user;

[0130] S22, text conversion is performed on the user voice to obtain corresponding user text;

[0131] S23, the user text is input into an intelligent dialogue robot to trigger the intelligent dialogue robot to perform the following steps:

[0132] When it is determined that the bottom guarantee response mechanism enabling condition is met, a preset response text configured for the service task is obtained, and the preset response text is used as dialogue text to respond to the user text; or,

[0133] When it is determined that the response enhancement condition is met, the knowledge base associated with the service task is queried according to the user text to obtain relevant external knowledge; and according to the external call knowledge and the user text, dialogue text is generated by using the built-in language model to respond to the user text.

[0134] In the above S21, the VAD technology in the external call communication system is used to detect the user voice of the target user to assist the collection of the user voice.

[0135] In the above S22, the ASR technology is used to realize the text conversion of the user voice.

[0136] In the above S23, the bottom guarantee response mechanism enabling condition includes that the intelligent dialogue robot (specifically, such as a language model built-in) is in a flow limiting state, such as the working load of the intelligent dialogue robot exceeding a preset load threshold. And the response enhancement condition includes but is not limited to that the user problem type is within a preset type range (such as query type) by keyword detection or semantic analysis on the user text, the user asks for specific service details (such as the target user asks for the specific preferential way of 618 promotion activities, the registration way, etc.), the user raises objections (such as worrying that the 618 promotion activities are fake, etc.). Under the response enhancement condition, the intelligent dialogue robot can use the built-in RAG model (retrieval enhancement model) to query the corresponding knowledge base according to the user text to obtain corresponding external knowledge, and combine the external knowledge with the user text to obtain enhanced information, and then use the built-in language model to generate and output dialogue text for responding to the user based on the enhanced information.

[0137] The knowledge base associated with the service task is created in advance, and stores knowledge information related to the service task (such as 618 promotion activity registration reminder). The knowledge information can include: core high-frequency QAs related to the service task, general QAs, etc. The core high-frequency QAs are formed by collecting and sorting the most frequently asked questions and their standard answers in the service task scenario. For example, by collecting and sorting the common questions and answers about 618 promotion activity registration reminders, a "core high-frequency QA" can be formed. The general QAs can cover common questions and answers across service tasks to improve the universality and flexibility of the knowledge base. In addition, the knowledge base can be created and updated through real-time intervention and offline backflow, etc.

[0138] Figure 4B The creation and update timing diagram of the knowledge base (vector table Igraph) realized through the online intervention link and the offline batch synchronization link is shown. As shown in Figure 4B As shown, the operator can perform knowledge import operation in an artificial manner. After the outbound call operation platform receives the knowledge imported by the operator, in addition to returning the import success message to the operator, it will also convert the imported knowledge into vector form through the Embeding service asynchronously, and the text vector obtained by conversion can be sent out through the swift message, realizing real-time synchronization to the vector table Igraph. After synchronization is completed, the outbound call operation platform returns the done signal to indicate that the synchronization has been completed. Further, high-frequency knowledge in the history chat can be clustered, and the high-frequency knowledge can be vectorized, and the text vector obtained after vectorization can be synchronized to the outbound call operation platform through mq, so that the outbound call operation platform inserts new knowledge through consuming mq, and synchronizes to the odps (Open Data Processing Service, an offline data processing service) in the next time period (t+1), and returns a synchronization success message to the outbound call operation platform after synchronization is completed. Further, the odps will synchronize all the full amount in it to the vector table Igraph.

[0139] Among them, the clustering of high-frequency knowledge in the history chat refers to the clustering analysis of the cleaned history chat records to extract high-frequency knowledge points. After cleaning the history chat records, the cleaned history chat records can be stored in a temporary table. The history chat records refer to the chat content (chat messages) recorded for the outbound call initiated for the corresponding service task. In addition, the mq synchronization refers to synchronizing data changes in the form of messages to the outbound call operation platform through the message queue. Consuming mq refers to the process of reading messages from the message queue, and inserting new knowledge refers to inserting the consumed new knowledge (new knowledge read from the message queue) to the local storage.

[0140] based on the combination Figure 4B Based on the description, the method provided by the embodiment can further include the following steps:

[0141] S31, in response to the knowledge import operation, obtaining the imported first knowledge data related to the service task, storing the first knowledge data in the knowledge base, and updating the knowledge base. For the specific implementation of this step S32, please refer to Figure 4B The related content shown in the online intervention link in the foregoing embodiment can be understood as a knowledge base update implemented through online intervention. And / or

[0142] S32, obtaining the interaction content of the current voice interaction record; extracting keywords from the interaction content according to the preset extraction condition, determining second knowledge data according to the extracted keywords, storing the second knowledge data in the knowledge base, and updating the knowledge base. Wherein, the interaction content includes text information corresponding to at least one round of voice dialogue; the text information includes dialogue text output by the intelligent robot and user text corresponding to user voice input by the target user. The preset extraction condition includes that the frequency of the keyword in the interaction content is greater than or equal to the preset frequency threshold (i.e. high-frequency keyword). For the specific implementation of this step S32, please refer to Figure 4B The related content shown in the offline batch synchronization link in the foregoing embodiment can be understood as a knowledge base update implemented through offline backflow.

[0143] Further, in the step 103, the dialogue text output by the intelligent dialogue robot is output in a streaming manner, and correspondingly, in the step 104, the dialogue text is also broadcast to the target user in a streaming manner, wherein the dialogue text generated by the stream is synthesized into voice through TTS technology, converted into corresponding dialogue voice, and then broadcast in a streaming manner. In addition, during the streaming broadcast process, it also supports strong humanization of tone, interactive interruption, etc. Strong humanization of tone means using high-quality text-to-speech (TTS) technology to make the broadcast tone of the dialogue text indistinguishable from that of a real person. Interactive interruption means that during the broadcast of the dialogue text output by the intelligent dialogue robot, the target user can interrupt and speak at any time, and the input voice of the target user can be immediately recognized and responded through the full-duplex outbound communication system.

[0144] Based on the foregoing content, in a specific implementable technical solution, the 104 "outputting the dialogue text to the target user" can include:

[0145] 1041, obtaining voice synthesis control parameters;

[0146] 1042. Perform voice synthesis on the dialogue text according to voice synthesis control parameters to obtain dialogue voice with humanized characteristics;

[0147] 1043. Transmit the dialogue voice to the target user.

[0148] In the above, the voice synthesis control parameters include but are not limited to at least one of the following: timbre parameters (such as male voice, female voice, hoarse voice), emotion parameters (for controlling the emotional expression of the voice, such as neutral, soft, etc.), in addition, it can also include pitch, speech rate, etc. According to the voice synthesis control parameters, high-quality TTS technology can be used to perform voice synthesis on the dialogue text, and the dialogue voice with humanized characteristics obtained after synthesis can be output to the target user in a streaming manner. Specifically, the dialogue voice is output to the client on the user side in a streaming manner, so that the client plays the dialogue voice to the target user in a streaming manner. The dialogue voice has humanized characteristics, which can make the dialogue voice sound almost the same as a real person when it is played subsequently, including timbre, intonation, emotion, tone change, etc.

[0149] In addition, the method provided by the embodiment can further include the following steps:

[0150] 105. Listen to the user voice of the target user while outputting the dialogue text to the target user in a streaming manner;

[0151] 106. If the user voice triggers the interrupt operation, interrupt the currently output dialogue text and respond to the user voice using the intelligent dialogue robot.

[0152] In the above, the dialogue text generated by streaming can be voice synthesized by TTS technology, converted into corresponding dialogue voice, and then played in a streaming manner. During the playing process, the user voice activity of the target user can be detected by VAD technology to assist in user voice collection, and the collected user voice can be recognized by ASR technology to obtain corresponding user text. If it is determined that the user text triggers the interrupt operation, the currently played dialogue text will be interrupted, wherein the interrupt operation includes but is not limited to detecting that the user text contains / semantically contains a specific keyword (such as query, etc.). For example, when the dialogue "Dear merchant, we will hold a 618 promotion activity. Please complete the registration before October 31..." is being played, and the merchant speaks "Wait a minute, please tell me the specific registration steps.", it is determined that the merchant triggers the playing interrupt operation, so the current playing content is stopped immediately. When the current playing content is stopped, the user voice is also responded to. The response mode is described in the foregoing embodiments in combination with Figure 6 the reply to the user's speaking content, which will not be specifically replied here.

[0153] Continuing to refer to Figure 6 When the conversation text content output by the intelligent dialogue robot hits the voice interaction end keyword (i.e., the LLM reply shown hits the session end keyword), the call will be hung up, and the voice interaction ends. At the same time, each round of conversation message in the voice interaction process is synchronized to the outbound operation platform for persistent storage, so as to be used subsequently, such as for optimizing each preset model (such as a language model) in the intelligent dialogue robot, for dialogue strategy optimization for the corresponding service task, for updating a knowledge base, etc.

[0154] Figure 7 An exemplary timing diagram of the automatic optimization link of the dialogue strategy (also referred to as a communication strategy) is shown. As seen in Figure 7, first is to carry out operation self-test, in this operation self-test stage, the operator can set the dialogue strategy for a service task (for a specific task instance in a service scene, such as 618 promotion activity registration reminder) self-test, self-test implementation includes: creating a self-test task on the outbound operation platform, this self-test task can be understood as a certain service task, and fill in the self-test person contact (as the contact of the called user in the outbound call), and import the previously set dialogue strategy and related environment variables (such as call time period). The outbound operation platform confirms that the import is successful and returns success to indicate that the import is completed. Further, the dialogue strategy will be converted into prompt word Prompt, after conversion, the self-test task is initiated and scheduling is started, the phone is dialed to realize voice interaction with the self-test personnel for a certain service task. Further, the operation cold start is carried out, in the operation cold start stage, the dialogue strategy is automatically tested and analyzed based on the selected users (such as merchants). Taking the selected user as the merchant as an example, the operation cold start stage includes: the operator creates a cold start task, the goal of this cold start task is to initiate outbound call to real merchants such as merchants for a certain service task, and will select merchants in the creation process (such as selecting "select 500 stores that have participated in "618 promotion activity" in the past year but have not yet registered for "618 promotion activity"); and, the dialogue strategy and related environment variables set for a certain service task are also imported, the dialogue strategy and related environment variables imported here are the same as the dialogue strategy and related environment variables imported in the self-test stage; then, the dialogue strategy will be converted into prompt word Prompt, after conversion, the cold start task is initiated and scheduling is started, the phone is dialed to each merchant selected to realize voice interaction with each merchant for a certain service task. According to the outbound call information (including telephone call state (such as connection, non-connection, rejection, etc.), communication content, etc.) recorded by the above outbound call to each merchant, the reason clustering analysis of outbound call failure is carried out, and the original dialogue strategy and outbound call failure reason are initiated to the relevant person in charge (the person in charge of the corresponding scene for the corresponding service task) for approval. If the approval result made by the person in charge is refused, the outbound operation platform receives the refusal message and does nothing. If the approval result made by the person in charge is agreed, the outbound operation platform receives the agreement message, updates the dialogue strategy, and initiates scheduling again for the updated dialogue strategy. Among them, the update of the dialogue strategy may be realized according to the outbound call failure reason obtained by analysis. The above automatic analysis and update of the dialogue strategy are realized online, and can also be realized by offline mode.

[0155] An embodiment of the present application also provides an intelligent voice interaction method. The method comprises the following steps:

[0156] 201、According to the e-commerce operation service task, determine the target merchant suitable for adaptation;

[0157] 202. Initiate a call to a target merchant using an intelligent conversational robot, conduct voice interaction with the target merchant using the intelligent conversational robot, and generate a conversation text for conveying e-commerce operation service information during the voice interaction; the e-commerce operation service information corresponds to the e-commerce operation service scenario;

[0158] 203. Output the conversation text to the target merchant.

[0159] In the above, in addition to screening out all eligible target merchants based on e-commerce operation service tasks (such as 618 promotion registration reminders), the e-commerce operation service will also be converted into an actual outbound call task, thereby using an intelligent dialogue robot to execute the outbound call task, thereby initiating calls to the target merchants.

[0160] For the specific implementation of the above 201 to 203, please refer to the relevant content in other embodiments.

[0161] In addition, the method provided in this embodiment may include other steps in addition to the steps given above. For details on the specific steps that may be included in this embodiment and the implementation of each step, please refer to the relevant content in other embodiments.

[0162] Another embodiment of the present application provides an intelligent voice interaction method. The method includes the following steps:

[0163] 301. Screen out suitable target merchants based on e-commerce operation service scenarios;

[0164] 302. Initiate a call to the target merchant using the intelligent conversational robot, conduct voice interaction with the target merchant using the intelligent conversational robot, and stream-output a conversation text for conveying e-commerce operation service information during the voice interaction; the e-commerce operation service information corresponds to the e-commerce operation service scenario;

[0165] 303. Streaming the conversation text to the target merchant;

[0166] 304. During the streaming output process, continuously monitor the target merchant's voice input;

[0167] 305. If a triggering interruption operation is detected, the output of the conversation text is interrupted, and the intelligent conversation robot responds to the voice input.

[0168] For the specific implementation of the above 301 to 305, please refer to the relevant content in other embodiments.

[0169] In addition to including the steps given above, the method provided in this embodiment can further include other steps. For specific steps that can be included in this embodiment and implementation of each step, refer to related content in other embodiments.

[0170] The application further provides an intelligent voice interaction method. The method comprises the following steps:

[0171] 401. According to the e-commerce operation service scenario, a target merchant is determined;

[0172] 402. A call is initiated to the target merchant by using an intelligent dialogue robot, so as to use the intelligent dialogue robot to perform voice interaction with the target merchant, and output dialogue text for conveying e-commerce operation service information in the voice interaction process; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario;

[0173] 403. The dialogue text is subjected to voice synthesis processing to obtain dialogue voice with personification characteristics;

[0174] 404. The dialogue voice is sent to a target user.

[0175] For specific implementation of the above 401-404, refer to related content in other embodiments.

[0176] In addition to including the steps given above, the method provided in this embodiment can further include other steps. For specific steps that can be included in this embodiment and implementation of each step, refer to related content in other embodiments.

[0177] It should be noted that in some of the processes described in the embodiments of the present application and the accompanying drawings, a plurality of operations appear in a specific order, but it should be clearly understood that these operations can be executed or in parallel without the order in which they appear in this document. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0178] The device embodiments corresponding to the methods provided by the present application will be introduced below.

[0179] An example embodiment of the present application provides a structural schematic diagram of a voice interaction device. The device comprises a determination creation module, a calling module, and an output module. The determination module is configured to determine a service task, and further configured to determine a target user requiring an outbound call and create a corresponding outbound call task according to the service task. The calling module is configured to call an intelligent dialogue robot constructed based on a preset model to perform the outbound call task, to use the intelligent dialogue robot to perform voice interaction with the target user, and to generate dialogue text for conveying service information by using the preset model in the voice interaction process; wherein the preset model comprises a language model, and the service information has a corresponding relationship with the service task. The output module is configured to output the dialogue text to the target user.

[0180] Optionally, the dialogue text is generated and broadcasted in a streaming manner. In addition, the device can further comprise a listening module and an interrupt / response module. The listening module is configured to listen to user voice of the target user while the dialogue text is broadcasted in a streaming manner. The interrupt / response module is configured to interrupt the output of the dialogue text and respond to the user voice by using the intelligent dialogue robot when a break operation is triggered according to the user voice.

[0181] Optionally, when the output module is used to output the dialogue text to the target user, it is specifically configured to: obtain voice synthesis control parameters; perform voice synthesis on the dialogue text according to the voice synthesis control parameters to obtain dialogue voice with personification characteristics; and send the dialogue voice to the target user; wherein the voice synthesis control parameters comprise at least one of the following: timbre parameters, emotion parameters.

[0182] Optionally, the device further comprises a collection module, a conversion module, and a triggering module. The collection module is configured to collect user voice of the target user in the voice interaction process. The conversion module is configured to convert the user voice into text to obtain corresponding user text. The triggering module is configured to input the user text into the intelligent dialogue robot to trigger the intelligent dialogue robot to perform the following steps: when it is determined that a bottom-line response mechanism enabling condition is met, obtain a preset response text configured for the service task; use the preset response text as dialogue text to respond to the user text; or when it is determined that a response enhancement condition is met, query a knowledge base associated with the service task according to the user text to obtain related external knowledge; and generate dialogue text by using the language model according to the external knowledge and the user text to respond to the user text.

[0183] Optionally, the device further comprises an acquisition module, a storage module, and an extraction / determination module. The acquisition module is configured to, in response to a knowledge import operation, acquire imported first knowledge data related to the service task, and store the first knowledge data in the knowledge base. The acquisition module is further configured to acquire interaction content of the voice interaction record. The extraction / determination module is configured to extract keywords from the interaction content according to a preset extraction condition, and determine second knowledge data according to the extracted keywords. The storage module is further configured to store the second knowledge data in the knowledge base. The interaction content includes text information corresponding to at least one round of voice dialogue, and the text information includes dialogue text output by the intelligent robot and user text corresponding to user voice input by the target user. The preset extraction condition includes that the frequency of the keywords in the interaction content is greater than or equal to a preset frequency threshold.

[0184] Optionally, the device further comprises a control module configured to control the outbound task. The trigger module is further configured to trigger the intelligent dialogue robot to perform the outbound task after the control. The control includes at least one of the following: the outbound task meets an admission rule, the communication resource configuration of the outbound task is completed, and the workload of the intelligent dialogue robot is less than a preset load threshold. The admission rule includes that the fatigue degree of the target user is less than a preset fatigue threshold, and the fatigue degree is determined according to the received outbound frequency.

[0185] An example embodiment of the present application further provides a structural schematic diagram of a voice interaction device. The device comprises a determination module, a calling module, and an output module. The determination module is configured to determine a matched target merchant according to an e-commerce operation service task. The calling module is configured to initiate a call to the target merchant by using an intelligent dialogue robot, to perform voice interaction between the target merchant and the intelligent dialogue robot, and to generate dialogue text for conveying e-commerce operation service information in the voice interaction process. The e-commerce operation service information has a corresponding relationship with the e-commerce operation service scene. The output module is configured to output the dialogue text to the target merchant.

[0186] The application further provides a structural diagram of a voice interaction device. The device comprises a determination module, a calling module, an output module, a listening module, and an interrupt / response module. The determination module is configured to determine a target merchant according to an e-commerce operation service scenario. The calling module is configured to initiate a call to the target merchant by using an intelligent dialogue robot, to perform voice interaction with the target merchant by using the intelligent dialogue robot, and to output dialogue text for conveying e-commerce operation service information in a streaming manner during the voice interaction; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario. The output module is configured to output the dialogue text to the target merchant in a streaming manner. The listening module is configured to continuously listen to voice input of the target merchant during the streaming output. The interrupt / response module is configured to interrupt the output of the dialogue text if a break operation is triggered, and to respond to the voice input by using the intelligent dialogue robot.

[0187] The application further provides a structural diagram of a voice interaction device. The device comprises a determination module, a calling module, an output module, a listening module, and an interrupt / response module. The determination module is configured to determine a target merchant according to an e-commerce operation service scenario. The calling module is configured to initiate a call to the target merchant by using an intelligent dialogue robot, to perform voice interaction with the target merchant by using the intelligent dialogue robot, and to output dialogue text for conveying e-commerce operation service information in a streaming manner during the voice interaction; the e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario. The output module is configured to output the dialogue text to the target merchant in a streaming manner. The listening module is configured to continuously listen to voice input of the target merchant during the streaming output. The interrupt / response module is configured to interrupt the output of the dialogue text if a break operation is triggered, and to respond to the voice input by using the intelligent dialogue robot.

[0188] It should be noted that the above devices can realize the technical solutions described in the above corresponding method embodiments, and the principles of the implementation of the above modules or units can be referred to the related content in the above corresponding method embodiments, which will not be described in detail here.

[0189] Figure 8 A block diagram of an electronic device for implementing the embodiments of the application is shown in FIG. 7. As shown in FIG. 7, the electronic device comprises a memory 71 and a processor 72. Figure 8 The memory 71 is configured to store computer programs and can be configured to store various data to support operations on the electronic device. Examples of the data include instructions of any application program or method for operating on the electronic device, data structures, contact data, phonebook data, messages, pictures, videos, etc.

[0190] The memory 71 is configured to store computer programs and can be configured to store various data to support operations on the electronic device. Examples of the data include instructions of any application program or method for operating on the electronic device, data structures, contact data, phonebook data, messages, pictures, videos, etc.

[0191] The processor 72 is coupled to the memory 71 and is configured to execute the computer program instructions stored in the memory 71 to implement the steps in the methods according to the embodiments of the present application.

[0192] Further, as shown in Figure 8 the electronic device further includes a communication component 73, a display 74, a power supply component 75, an audio component 76, and other components. Figure 8 Some components are only schematically shown in the figures, and it does not mean that the electronic device only includes Figure 8 the components shown in the figures. In addition, Figure 8 The components in the dashed box in the figures are optional components, rather than mandatory components, and the specific components can be determined according to the product form of the electronic device. The electronic device in the embodiment can be implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, or an IOT device, or can be implemented as a server device such as a general server, a cloud server, or a server array, or can be implemented as an integrated device of the terminal device and the server device, etc. If the electronic device in the embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, or a smart phone, it can include the components in the dashed box in the figures; if the electronic device in the embodiment is implemented as a server device such as a general server, a cloud server, or a server array, it can not include the components in the dashed box in the figures. Figure 8 Figure 7

[0193] The above memory can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0194] The above communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a 2G, 3G, 4G / LTE, 5G, or the like mobile communication network, or a combination thereof. In an example embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0195] ​​The display includes a screen, which can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding movement, but also detect a duration and a pressure associated with the touching or the sliding movement.

[0196] The power component provides power to various components of the device in which the power component is located. The power component can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which the power component is located.

[0197] The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) that is configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in a memory or transmitted via the communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0198] Accordingly, the embodiments of the present application also provide a computer readable storage medium storing a computer program, when the computer program is executed by a processor, the processor is enabled to implement each step in the above-mentioned method embodiments. The computer readable storage medium includes volatile or non-volatile or a combination thereof, and can be removable or non-removable. Examples of the computer readable storage medium include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium

[0199] Accordingly, the embodiments of the present application also provide a computer program product, which comprises computer programs or instructions, and when the computer programs or instructions are executed by a processor, the processor can realize each step in the above-mentioned method embodiments. It should be understood that each process or a combination of multiple processes in the above-mentioned method flow can be realized by the computer programs or instructions. In addition, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices, so that the processor of the general-purpose computer, the special-purpose computer, the embedded processor or other programmable data processing devices can be realized as an apparatus for realizing the corresponding functions in the above-mentioned method embodiments.

[0200] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed, or further include elements inherent in such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0201] The above only describes the embodiments of the present application and is not intended to limit the present application. The present application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. An intelligent voice interaction method, characterized in that: include: Determine service tasks; According to the service task, determine the target users who need outbound calls and create corresponding outbound call tasks; Invoking an intelligent conversational robot built based on a preset model to execute the outbound call task, so as to use the intelligent conversational robot to conduct voice interaction with the target user, and using the preset model to generate a dialogue text for conveying service information during the voice interaction; wherein the preset model includes a language model, and the service information has a corresponding relationship with the service task; The conversation text is output to a target user.

2. The method according to claim 1, characterized in that The dialogue text is generated and output in a streaming manner; The method further comprises: While streaming the conversation text, monitoring the user voice of the target user; When an interruption operation is triggered according to the user voice, the output of the conversation text is interrupted, and an intelligent conversation robot is used to respond to the user voice.

3. The method according to claim 1, characterized in that Outputting the conversation text to the target user includes: Get speech synthesis control parameters; Performing speech synthesis on the dialogue text according to speech synthesis control parameters to obtain dialogue speech with anthropomorphic characteristics; Sending the conversation voice to the target user; The speech synthesis control parameters include at least one of the following: a timbre parameter and an emotion parameter.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: During the voice interaction process, the target user's voice is collected; Converting the user's voice into text to obtain corresponding user text; Input the user text into the intelligent dialogue robot, triggering the intelligent dialogue robot to execute the following steps: When it is determined that the minimum response mechanism activation condition is met, obtaining a preset response text configured for the service task; using the preset response text as a dialogue text to respond to the user text; or, When it is determined that the response enhancement conditions are met, the knowledge base associated with the service task is queried based on the user text to obtain relevant external knowledge; based on the external knowledge and the user text, the language model is used to generate a dialogue text to respond to the user text.

5. The method according to claim 4, characterized in that Also includes: In response to the knowledge import operation, obtaining imported first knowledge data related to the service task; storing the first knowledge data in the knowledge base; and / or Obtaining interaction content for the voice interaction record; According to preset extraction conditions, key words are extracted from the interactive content, and second knowledge data is determined based on the extracted key words, and the second knowledge data is stored in the knowledge base; wherein, the interactive content includes text information corresponding to at least one round of voice dialogue, and the text information includes the dialogue text output by the intelligent robot and the user text corresponding to the user voice input by the target user; the preset extraction conditions include: the frequency of occurrence of a word in the interactive content is greater than or equal to a preset frequency threshold.

6. The method according to any one of claims 1 to 3, characterized in that Also includes: Managing and controlling the outbound call tasks; After the control is passed, the outbound call task is triggered based on the intelligent dialogue robot; Among them, the control includes at least one of the following: the outbound call task meets the access rules, the communication resource configuration of the outbound call task is completed, and the workload of the intelligent dialogue robot is less than the preset load threshold; the access rules are met, including the target user's fatigue being less than the preset fatigue threshold, and the fatigue is determined according to the frequency of the outbound calls received.

7. An intelligent voice interaction method, characterized in that: include: Determine the appropriate target merchants based on the e-commerce operation service scenarios; Initiate a call to the target merchant using an intelligent conversational robot, conduct voice interaction with the target merchant using the intelligent conversational robot, and stream output a conversation text for conveying e-commerce operation service information during the voice interaction; The e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario; Streaming the conversation text to the target merchant; During the streaming output process, continuously monitor the target merchant's voice input; If an interruption operation is detected, the output of the conversation text is interrupted, and the intelligent conversation robot responds to the voice input.

8. An intelligent voice interaction method, characterized in that: include: Determine the appropriate target merchants based on the e-commerce operation service scenarios; Initiate a call to the target merchant using an intelligent conversational robot, conduct voice interaction with the target merchant using the intelligent conversational robot, and output a conversation text for conveying e-commerce operation service information during the voice interaction process; The e-commerce operation service information has a corresponding relationship with the e-commerce operation service scenario; Performing speech synthesis processing on the dialogue text to obtain dialogue speech with anthropomorphic characteristics; The conversation voice is sent to a target merchant.

9. An intelligent voice interaction system, characterized in that: include: The server is deployed with an intelligent dialogue robot, which is built based on a preset model, including a language model; The server is used to determine the service task; according to the service task, determine the adapted target user and create the corresponding outbound call task; Invoking the intelligent conversational robot to execute the outbound call task, using the intelligent conversational robot to perform voice interaction with the target user, and using the preset model to output a conversation text for conveying service information during the voice interaction process; The service information and the service task have a corresponding relationship; The client is used to output the conversation text to the target user; The service tasks include e-commerce operation service tasks, and the target users include target merchants.

10. An electronic device, characterized in that: including a memory and a processor; wherein, The memory is used to store executable instructions; The processor implements the steps in the intelligent voice interaction method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium, characterized in that The storage medium stores computer instructions, which, when executed by a processor, can implement the steps of the intelligent voice interaction method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor executes the steps in the intelligent voice interaction method according to any one of claims 1 to 8.

Images