A dialogue text generation method, device, medium and electronic equipment

Abstract

This specification provides a method, apparatus, medium, and electronic device for generating dialogue text. In this method, dialogue state data of the current conversation is first acquired, including at least one of the following: historical dialogue records between customer service and the user, and user attribute data. The dialogue state data is then input into a preset generation model to obtain candidate dialogue texts. Each candidate dialogue text is evaluated to determine a first reward value reflecting the estimated response rate after the user receives the text and a second reward value reflecting the accuracy of the text's content. These two values ​​are then combined to obtain a comprehensive reward value that comprehensively reflects the attractiveness and accuracy of the candidate dialogue text to the user. Finally, a target dialogue text is selected from the candidate dialogue texts based on the comprehensive reward value for use in conversations with the user, thereby improving user recall and conversion rates.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of large model reasoning technology, and more particularly to a method, apparatus, medium and electronic device for generating dialogue text. Background Technology

[0002] Currently, in various intelligent customer service scenarios, such as insurance recommendation scenarios, users may stop actively interacting with the business platform or not responding to various information pushed by the platform because the recommended content does not meet their needs or the dialogue interaction experience is poor. In order to improve the conversion rate of these users, the business platform can proactively send some dialogue texts to users to guide them to reply and reactivate their willingness to interact.

[0003] The dialogue text that the business platform actively sends to users is usually generated by a large language model. However, due to the poor quality of the dialogue text generated by the large language model, the actual user recall rate is low after the business platform actively sends the dialogue text generated by the large language model to users. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a method for generating dialogue text is proposed, comprising: Obtain the dialogue status data of the current conversation; the dialogue status data includes at least one of the historical dialogue records between the customer service representative and the user involved in the current conversation and the user's attribute data; The dialogue state data is input into a preset generation model to obtain candidate dialogue texts; Each candidate dialogue text is evaluated to determine a first reward value and a second reward value corresponding to each candidate dialogue text. The first reward value and the second reward value are then combined to obtain a comprehensive reward value. The first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text. The second reward value is used to reflect the accuracy of the content of the candidate dialogue text. Based on the comprehensive reward value, a target dialogue text is selected from the candidate dialogue texts for use in a dialogue with the user.

[0005] According to a second aspect of one or more embodiments of this specification, a dialogue text generation apparatus is provided, comprising: The acquisition module is used to acquire the dialogue status data of the current conversation; the dialogue status data includes the historical dialogue records between the customer service representative and the user involved in the current conversation, as well as the user's attribute data; The generation module is used to input the dialogue state data into a preset generation model to obtain candidate dialogue texts. The determination module is used to evaluate each candidate dialogue text to determine a first reward value and a second reward value corresponding to each candidate dialogue text, and to merge the first reward value and the second reward value to obtain a comprehensive reward value; the first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text; the second reward value is used to reflect the professionalism and compliance of the candidate dialogue text. The selection module is used to select a target dialogue text from the candidate dialogue texts based on the comprehensive reward value, for use in dialogue with the user.

[0006] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor performs the executable instructions to implement the steps of the dialogue text generation method described above.

[0007] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the dialogue text generation method described above.

[0008] According to a fifth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the above-described dialog text generation method.

[0009] In this method, the current dialogue state data is obtained, which includes at least one of the historical dialogue records between customer service and user and user attribute data. The dialogue state data is then input into a preset generation model to obtain candidate dialogue texts. Each candidate dialogue text is evaluated to determine a first reward value that reflects the estimated response rate after the user receives the text and a second reward value that reflects the accuracy of the text content. The two are then combined to obtain a comprehensive reward value. Finally, a target dialogue text is selected from the candidate dialogue texts based on the comprehensive reward value for dialogue with the user.

[0010] As can be seen from the above methods, a dual reward mechanism can be adopted, namely, a first reward value that reflects the user's willingness to speak after receiving the dialogue text, and a second reward value that ensures the accuracy of the generated dialogue text content. This balances the contradiction between the attractiveness of the dialogue text to the user and the standardization of the text, thereby effectively improving the overall quality of the final selected target dialogue text and thus improving the actual user recall rate. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the architecture of an intelligent customer service system provided in an exemplary embodiment.

[0012] Figure 2 This is a flowchart illustrating a dialogue text generation method provided in an exemplary embodiment.

[0013] Figure 3 This is a schematic diagram of a predicted response reward model provided in an exemplary embodiment.

[0014] Figure 4 This is a schematic diagram of a model training process provided in an exemplary embodiment.

[0015] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment.

[0016] Figure 6 This is a block diagram of a dialogue text generation apparatus provided in an exemplary embodiment. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0018] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0019] Currently, in various intelligent customer service scenarios, such as insurance recommendation scenarios, users may stop actively interacting with the business platform or not responding to various information pushed by the platform because the recommended content does not meet their needs or the dialogue interaction experience is poor. In order to improve the conversion rate of these users, the business platform can proactively send some dialogue texts to users to guide them to reply and reactivate their willingness to interact.

[0020] The dialogue texts that business platforms proactively send to users are usually generated by large language models. However, since large language models are usually trained with text fluency or general human preferences as training objectives, some dialogue texts generated by large language models have obvious defects. That is, the dialogue texts may contain exaggerated claims, fail to meet industry compliance requirements, or have imprecise expressions, or the texts may be stiff, lack targeting, and fail to match the potential needs and interaction preferences of different users. This can easily lead to user aversion and refusal to respond to platform pushes, resulting in a low actual user recall rate.

[0021] Based on this, this specification provides a method for generating dialogue text that employs a dual reward mechanism. This mechanism balances the contradiction between the attractiveness of the dialogue text to users and the standardization of the text by using a first reward value that reflects the user's willingness to speak upon receiving the dialogue text and a second reward value that ensures the accuracy of the generated dialogue text. This effectively improves the overall quality of the final selected target dialogue text and, consequently, increases the actual user recall rate.

[0022] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0023] Figure 1 This is a schematic diagram of the architecture of an intelligent customer service system provided in an exemplary embodiment. For example... Figure 1 As shown, the system may include a server 11, a network 12, and several electronic devices, such as a PC (Personal Computer) 13, a mobile phone 14, etc.

[0024] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted in a host cluster. During operation, server 11 can run server-side programs for a certain application to implement the relevant functions of that application. For example, when server 11 runs the intelligent customer service program of the application, it can be implemented as a corresponding intelligent customer service platform and used to execute the dialogue text generation method provided in this specification.

[0025] PC13 and mobile phone14 are just some of the types of electronic devices that users can use. In reality, users can obviously also use electronic devices such as tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments in this specification do not limit this. During operation, the electronic device can run a client-side program of an application to achieve the relevant functions of that application. For example, when the electronic device runs a smart customer service program, it can act as a client for the corresponding smart customer service. The aforementioned smart customer service client application can be launched and run on the electronic device. This client-side program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. Of course, when using web technologies such as HTML5 or similar, the relevant functions can be achieved through a page displayed by a browser. This browser can be a standalone browser application or a browser module embedded in some applications.

[0026] As for the network 12 that enables interaction between electronic devices such as PC13 and mobile phone 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, PC13 can support both wired and wireless communication, so it can use either wired or wireless networks as needed. Mobile phone 14 typically only supports wireless communication, so it can use a wireless network for communication.

[0027] Figure 2 This is a flowchart illustrating a dialogue text generation method provided in an exemplary embodiment, including: S200: Obtain the dialogue status data of the current conversation; the dialogue status data includes the historical dialogue records between the customer service representative and the user involved in the current conversation, as well as the user's attribute data.

[0028] In this specification, the executing entity for implementing the dialogue text generation method can refer not only to computing devices located in the backend of a business application, such as servers and cloud computing clusters, but also to terminal devices with data processing capabilities, such as personal computers and mobile terminals. For ease of description, the following explanation uses a server as the executing entity.

[0029] In practical applications, when the server detects that a user meets the preset high churn risk conditions, the user can be identified as a high churn risk user. The server can then obtain the current conversation status data between the customer service representative and the user. Based on the obtained conversation status data, the server can generate target conversation text and proactively send the target conversation text to the user through the current conversation to guide the user to reply, reactivate the user's willingness to interact, and improve user recall rate and business conversion rate.

[0030] The aforementioned high churn risk conditions can be set according to actual needs. For example, if it is determined that a user has not interacted with the business platform or customer service within a specified time period, it can be considered as meeting the preset high churn risk conditions. The specified time period can be set according to actual needs.

[0031] For example, if a user's feedback frequency regarding content pushed by the business platform is lower than a preset frequency threshold, it can be considered as meeting the preset high churn risk condition. This frequency threshold can be set according to actual needs.

[0032] The aforementioned dialogue status data may include at least one of the following: historical dialogue records between the customer service representative and the user involved in the current dialogue, as well as user attribute data, which are used to characterize the user's interaction status and personalized features.

[0033] The aforementioned customer service can refer to either a human customer service representative who is currently interacting with the user in the conversation, or an automated customer service system that is currently interacting with the user in the conversation.

[0034] The aforementioned current dialogue refers to the historical dialogue records between customer service and the user within a specified time period.

[0035] The aforementioned user attribute data can refer to data used to characterize user identity features, needs features, and behavioral features, such as: user's age, gender, assets, behavioral sequence, business preferences, user tags, etc.

[0036] Of course, in practical applications, the server can also generate corresponding target dialogue texts in real time or on demand for users other than those at high churn risk, and proactively send the target dialogue texts to those users. In this case, since there may be new users of the business platform among these users, the dialogue state data of the current conversation obtained by the server may only contain the user's attribute data.

[0037] S202: Input the dialogue state data into a preset generation model to obtain each candidate dialogue text.

[0038] Furthermore, the server can input the acquired dialogue state data into a preset generation model to obtain candidate dialogue texts.

[0039] The aforementioned candidate dialogue text can refer to dialogue text generated based on the current user's dialogue state data, which can be actively sent by the server to the user to guide the user's reply and reactivate the user's willingness to interact.

[0040] The aforementioned generative models can be large language models (LLMs) such as Generative Pre-trained Transformer (GPT) and Text-to-Text Transfer Transformer (T5).

[0041] In real-world scenarios, the server can input the acquired dialogue state data into the generative model multiple times, so that the generative model can generate multiple times based on the dialogue state data to obtain various candidate dialogue texts.

[0042] For example, when the server first inputs the acquired dialogue state data into the generative model, the generative model generates the first candidate dialogue text. Then, the server inputs the acquired dialogue state data into the generative model a second time, the generative model generates the second candidate dialogue text, and so on.

[0043] Of course, the server can also construct generation prompts based on the dialogue state data. These generation prompts are used to guide the generation model to generate multiple candidate dialogue texts that meet business requirements in batches based on the dialogue state data. Then, the generation prompts can be input into the generation model so that the generation model can generate text based on the generation prompts, thereby obtaining each candidate dialogue text.

[0044] In addition, multiple generative models can be deployed on the server. In this case, the server can input the dialogue state data into each generative model to obtain each candidate dialogue text.

[0045] In addition, in some intelligent customer service scenarios, such as insurance recommendation business scenarios, the different business platforms have different requirements for the wording, expected style, and professional knowledge of the business objects involved in the dialogue text sent by the customer service to the user. Therefore, the dialogue text sent by the customer service to the user also needs to meet the specific requirements of different business platforms.

[0046] Based on this, the aforementioned dialogue state data can also include basic constraint information, so that the generation model can generate candidate dialogue text suitable for the corresponding business scenario based on the basic constraint information of different business platforms.

[0047] The aforementioned business objects can refer to specific products or services provided by the business platform as mentioned in the dialogue text, such as: insurance products, wealth management products, credit services, membership benefits, and products to be recommended.

[0048] The aforementioned basic constraint information can be used to limit the key elements that must be included in the candidate dialogue text, the prohibited expressions, and the specific wording structures that must be followed. For example, in the insurance recommendation scenario, the basic constraint information can require that the generated dialogue text must include compliant content such as an explanation of insurance liability, a reminder of exclusion clauses, payment methods, and cooling-off period rights, while prohibiting the use of abnormal language such as promising returns or exaggerating the scope of coverage.

[0049] S204: Evaluate each candidate dialogue text to determine a first reward value and a second reward value corresponding to each candidate dialogue text, and merge the first reward value and the second reward value to obtain a comprehensive reward value; the first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text; the second reward value is used to reflect the accuracy of the content of the candidate dialogue text.

[0050] Furthermore, after obtaining each candidate dialogue text, the server can evaluate each candidate dialogue text to determine the first reward value and the second reward value corresponding to that candidate dialogue text.

[0051] The first reward value mentioned above is used to reflect the estimated response rate of the user after receiving the candidate dialogue text.

[0052] The second reward value mentioned above is used to reflect the accuracy of the content of the candidate dialogue text.

[0053] For ease of understanding, the following sections provide detailed explanations of the methods for determining the first reward value and the second reward value.

[0054] For the first reward value, the server can perform feature extraction on each candidate dialogue text to obtain the text feature representation corresponding to the candidate dialogue text, and perform feature extraction on the acquired dialogue state data to obtain the dialogue feature representation of the current dialogue. Then, based on the similarity between the text feature representation and the dialogue feature representation, the server can determine the first reward value corresponding to the candidate dialogue text.

[0055] Specifically, for each candidate dialogue text, the server can determine the user's estimated response rate based on the similarity between the text feature representation of the candidate dialogue text and the dialogue feature representation of the current dialogue, and then determine the first reward value corresponding to the candidate dialogue text based on the estimated response rate.

[0056] The user's estimated response rate is positively correlated with the similarity between the text feature representation of the candidate dialogue text and the dialogue feature representation of the current dialogue, and the estimated response rate is positively correlated with the first reward value.

[0057] Furthermore, in practical applications, since candidate dialogue text and dialogue state data are heterogeneous data with different modalities and structures, their distributions differ significantly. If a single encoding model is used, it is difficult to simultaneously encode effective and aligned feature representations for both. However, the dual-tower model can independently encode inputs with different structures and then perform similarity calculations through a unified feature space, thereby improving the accuracy of the predicted opening rate determined based on the final calculated similarity.

[0058] Therefore, in this specification, the server may also deploy a predicted response reward model employing a dual-tower architecture, wherein the predicted response reward model may include a dialogue text encoding tower and a state data encoding tower, as detailed below. Figure 3 As shown.

[0059] Figure 3 This is a schematic diagram of a predicted response reward model provided in an exemplary embodiment.

[0060] Combination Figure 3 As can be seen, the server can input each candidate dialogue text into the dialogue text encoding tower of the preset prediction response reward model to obtain the text feature representation corresponding to the candidate dialogue text, and can also input the obtained dialogue state data into the state data encoding tower of the prediction response reward model to obtain the dialogue feature representation of the current dialogue.

[0061] It should be noted that the aforementioned dialogue text encoding tower can be composed of a large language model. In this case, the server can use the large language model contained in the aforementioned dialogue text encoding tower to encode the input candidate dialogue text and obtain the text feature representation corresponding to the candidate dialogue text.

[0062] Furthermore, as can be seen from the above, the user status data obtained by the server can include structured user attribute data and unstructured historical behavior sequences. Therefore, the dialogue text encoding tower of the response reward prediction model can include a feature embedding module and a sequence encoding module, wherein the sequence encoding module can be composed of a large language model.

[0063] Based on this, the server can use the feature embedding module of the dialogue text encoding tower to extract features from structured data such as age, gender, ..., assets, to obtain the corresponding first sub-feature representation. In addition, it can use the sequence encoding module contained in the dialogue text encoding tower to encode behavioral sequences and historical dialogue sequences to obtain the second sub-feature representation. The first sub-feature representation and the second sub-feature representation are then fused to obtain the dialogue feature representation of the current dialogue.

[0064] It should be noted that the aforementioned predicted response reward model can only be deployed to the server after training. The method for training the predicted response reward model can be as follows: obtain real dialogue state data and real dialogue text sent to the user based on the real dialogue state data, input the real dialogue state data and the real dialogue text into the predicted response reward model to be trained, obtain the first reward value corresponding to the real dialogue text, and determine the predicted result of whether the user will reply after receiving the real dialogue text based on the first reward value corresponding to the real dialogue text. Then, determine the loss based on the deviation between the predicted result and the actual result of whether the user will reply after receiving the real dialogue text, and train the predicted response reward model to be trained with the goal of minimizing the loss, to obtain the trained predicted response reward model.

[0065] For the second reward value, the server can generate a prompt text corresponding to each candidate dialogue text based on that candidate dialogue text. The prompt text is used to guide the preset accuracy reward model to determine the second reward value corresponding to the candidate dialogue text according to the preset evaluation dimensions.

[0066] Furthermore, the server can input the prompt text into the accuracy reward model to obtain the second reward value corresponding to the candidate dialogue text.

[0067] The aforementioned evaluation dimensions may include at least one of the following: compliance dimension, which characterizes the satisfaction rate of the candidate dialogue text with each preset expression goal; authenticity dimension, which characterizes the information authenticity of the candidate dialogue text; compliance dimension, which characterizes whether the candidate dialogue text conforms to preset risk control rules; and expressive dimension, which characterizes the language fluency of the candidate dialogue text.

[0068] Furthermore, after determining the first reward value and the second reward value, the server can combine the first reward value and the second reward value to obtain a comprehensive reward value, as shown in the following formula: In the above formula, For the overall reward value, Let i be the i-th candidate dialogue text among all candidate dialogue texts. The first reward value, As the second reward value, and This is a weighting coefficient, which can be determined based on the actual application scenario. For example, in a product promotion scenario, the business platform may place more emphasis on the user's response rate; in this case, the weighting coefficient will be set accordingly. It can be 0.3. It can be 0.7. For example, in a microloan scenario, the business platform might prioritize the accuracy of the dialogue text; in this case, the setting... It can be 0.8. It can be 0.2, etc.

[0069] In practical applications, since the initial comprehensive reward value is obtained by simply weighting and fusing the first reward value and the second reward value, it is difficult to reflect the relative advantages between the candidate dialogue texts. If the overall quality of the candidate texts is too high or too low, it is easy to make it impossible to accurately distinguish between the best and second-best texts, thus affecting the accuracy of the target dialogue text selection. Therefore, the server can also determine the initial comprehensive reward value of each candidate dialogue text based on the first reward value and the second reward value. Then, the comprehensive reward value of the candidate dialogue text can be determined based on the mean and / or standard deviation of the initial comprehensive reward values ​​of each candidate dialogue text.

[0070] The aforementioned comprehensive reward value is used to reflect the degree of advantage of the initial comprehensive reward value of the candidate dialogue text relative to the initial comprehensive reward values ​​of other candidate dialogue texts.

[0071] Specifically, the aforementioned initial comprehensive reward value can be obtained by weighting and fusing the first reward value and the second reward value according to the above method.

[0072] The method for determining the comprehensive reward value of a candidate dialogue text based on the mean and / or standard deviation of the initial comprehensive reward values ​​of each candidate dialogue text can refer to the following formula: In the above formula, This is the total reward value for the i-th candidate dialogue text. This is the initial comprehensive reward value for the i-th candidate dialogue text. This is the average of the initial combined reward values ​​for all candidate dialogue texts. This is the standard deviation of the initial combined reward value of all candidate dialogue texts. It is a constant used to prevent division by zero.

[0073] Furthermore, in practical applications, the aforementioned generative model needs to be trained before it can be deployed to a server to generate candidate dialogue text, specifically as follows: Figure 4 As shown.

[0074] Figure 4 This is a schematic diagram of a model training process provided in an exemplary embodiment.

[0075] Combination Figure 4 As can be seen, the server can input the preset sample dialogue state data into the preset initial generation model to generate candidate dialogue texts for each sample. Then, it can determine the comprehensive reward value corresponding to each candidate dialogue text. Based on the comprehensive reward value corresponding to each candidate dialogue text, the server can select the target dialogue text from the candidate dialogue texts. Based on the difference between the comprehensive reward value of the target dialogue text and the comprehensive reward value of other candidate dialogue texts, the server can determine the target loss. With the minimum target loss as the optimization objective, the server can train the preset initial generation model to obtain the generation model.

[0076] Specifically, the target loss mentioned above is positively correlated with the difference between the comprehensive reward value of the selected target dialogue text and the comprehensive reward value of other candidate dialogue texts. This can be understood as follows: the larger the comprehensive reward value of the determined target dialogue text relative to the comprehensive reward values ​​of other candidate dialogue texts, the smaller the determined target loss.

[0077] S206: Based on the comprehensive reward value, select the target dialogue text from the candidate dialogue texts for dialogue with the user.

[0078] Specifically, the server can select the candidate dialogue text with the highest comprehensive reward value from all candidate dialogue texts as the target dialogue text, based on the comprehensive reward value corresponding to each candidate dialogue text.

[0079] If multiple candidate dialogue texts have the same comprehensive reward value and are all the highest, then the final target dialogue text can be determined by combining the preset auxiliary filtering rules.

[0080] The above-mentioned auxiliary filtering rules can be set according to the actual business scenario requirements. For example, candidate dialogue texts with a text length lower than a preset threshold can be selected as target dialogue texts.

[0081] Furthermore, after determining the target dialogue text, the server can proactively send the target dialogue text to the corresponding user through the interaction channel corresponding to the current dialogue (such as intelligent customer service dialog box, SMS, APP push, etc.) to guide the user to reply, reactivate the user's willingness to interact, and thus improve the user recall rate and conversion rate.

[0082] Meanwhile, the server can also store the selected target dialogue text, each candidate dialogue text and its corresponding comprehensive reward value, evaluation data, etc., for subsequent iterative optimization of the generation model and the predicted response reward model.

[0083] As can be seen from the above, the server can effectively improve the overall quality of the final selected target dialogue text by adopting a dual reward mechanism. This mechanism uses a first reward value to reflect the user's willingness to speak upon receiving the dialogue text, and a second reward value to ensure the accuracy of the generated dialogue text. This balances the contradiction between the attractiveness of the dialogue text to the user and the standardization of the text, thereby improving the actual user recall rate.

[0084] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 5 As shown, device 500 mainly consists of a communication interface 502, a user interface 504, a processor 506, and a data storage 508. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 510. The communication interface 502 enables device 500 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 502 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 502 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 502 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0085] User interface 504 includes receiving user input and providing output to the user. Therefore, user interface 504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 504 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 504 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 500 may support remote access from other devices via communication interface 502 or another physical interface (not shown). User interface 504 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 504 may also be configured as a display device for rendering or displaying text fragments.

[0086] Processor 506 may contain one or more general-purpose processors and / or special-purpose processors.

[0087] Data storage 508 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 506. Data storage 508 may include removable and non-removable components.

[0088] Processor 506 is capable of executing program instructions 518 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 508 to perform the various functions described herein. Data storage 508 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 500, enable device 500 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 518 by processor 506 may result in processor 506 using data 512.

[0089] For example, program instructions 518 may include an operating system 522 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 500 and one or more applications 520 (e.g., a browser, social application, or game application). Similarly, data 512 may include operating system data 516 and application data 514. Operating system data 516 is primarily accessible to the operating system 522, while application data 514 is primarily accessible to one or more applications 520. Application data 514 may reside in a file system visible or hidden from the user of device 500.

[0090] Application 520 can communicate with operating system 522 through one or more application programming interfaces (APIs). These APIs help application 520 read and / or write application data 514, transmit or receive information via communication interface 502, receive or display information on user interface 504, etc.

[0091] In some terminology, application 520 may be simply referred to as "app". Furthermore, application 520 can be downloaded to device 500 through one or more online app stores or app markets. However, applications can also be installed on device 500 in other ways, such as through a web browser or a physical interface on device 500 (e.g., a USB port).

[0092] Please refer to Figure 6 Dialogue text generation devices can be applied to, for example Figure 5 The device shown implements the technical solution of this specification. The dialogue text generation device may include: The acquisition module 601 is used to acquire the dialogue status data of the current dialogue; the dialogue status data includes the historical dialogue records between the customer service representative and the user involved in the current dialogue and the user's attribute data; The generation module 602 is used to input the dialogue state data into a preset generation model to obtain each candidate dialogue text; The determination module 603 is used to evaluate each candidate dialogue text to determine a first reward value and a second reward value corresponding to each candidate dialogue text, and to merge the first reward value and the second reward value to obtain a comprehensive reward value; the first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text; the second reward value is used to reflect the professionalism and compliance of the candidate dialogue text. The selection module 604 is used to select a target dialogue text from the candidate dialogue texts based on the comprehensive reward value, so as to conduct a dialogue with the user.

[0093] Optionally, the determining module 603 is specifically configured to: extract features from each candidate dialogue text to obtain a text feature representation corresponding to the candidate dialogue text; extract features from the dialogue state data to obtain a dialogue feature representation of the current dialogue; and determine a first reward value corresponding to the candidate dialogue text based on the similarity between the text feature representation and the dialogue feature representation.

[0094] Optionally, the determining module 603 is specifically used to: input each candidate dialogue text into the dialogue text encoding tower of the preset estimated response reward model to obtain the text feature representation corresponding to the candidate dialogue text; and input the dialogue state data into the state data encoding tower of the estimated response reward model to obtain the dialogue feature representation of the current dialogue.

[0095] Optionally, the determining module 603 is specifically used to: determine the user's estimated response rate based on the similarity between the text feature representation and the dialogue feature representation; wherein the estimated response rate is positively correlated with the similarity; and determine the first reward value corresponding to the candidate dialogue text based on the estimated response rate.

[0096] Optionally, the determining module 603 is specifically used to generate a prompt word text corresponding to each candidate dialogue text; the prompt word text is used to guide a preset accuracy reward model to determine the second reward value corresponding to the candidate dialogue text according to preset evaluation dimensions; and the prompt word text is input into the accuracy reward model to obtain the second reward value corresponding to the candidate dialogue text.

[0097] Optionally, the evaluation dimensions include at least one of the following: a compliance dimension for characterizing the satisfaction rate of the candidate dialogue text with each preset expression goal; an authenticity dimension for characterizing the information authenticity of the candidate dialogue text; a compliance dimension for characterizing whether the candidate dialogue text conforms to preset risk control rules; and an expressive dimension for characterizing the language fluency of the candidate dialogue text.

[0098] Optionally, the determining module 603 is specifically configured to: for each candidate dialogue text, determine an initial comprehensive reward value for the candidate dialogue text based on the first reward value and the second reward value; determine a comprehensive reward value for the candidate dialogue text based on the mean and / or standard deviation of the initial comprehensive reward values ​​of the candidate dialogue texts; the comprehensive reward value is used to reflect the degree of advantage of the initial comprehensive reward value of the candidate dialogue text relative to the initial comprehensive reward values ​​of other candidate dialogue texts.

[0099] Optionally, the device further includes: a training module 605; The training module 605 is specifically used to: input preset sample dialogue state data into a preset initial generation model to generate candidate dialogue texts for each sample; determine the comprehensive reward value corresponding to each candidate dialogue text; select a target dialogue text from the candidate dialogue texts based on the comprehensive reward value corresponding to each candidate dialogue text; determine the target loss based on the difference between the comprehensive reward value of the target dialogue text and the comprehensive reward value of other candidate dialogue texts; the target loss is positively correlated with the difference; and train the preset initial generation model with the minimum target loss as the optimization objective to obtain the generation model.

[0100] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0101] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0102] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0103] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0104] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0105] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0106] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0107] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0108] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0109] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0110] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for generating dialogue text, comprising: Get the current conversation state data; The dialogue status data includes at least one of the following: historical dialogue records between the customer service representative and the user involved in the current dialogue, and the user's attribute data. The dialogue state data is input into a preset generation model to obtain candidate dialogue texts; Each candidate dialogue text is evaluated to determine a first reward value and a second reward value corresponding to each candidate dialogue text. The first reward value and the second reward value are then combined to obtain a comprehensive reward value. The first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text. The second reward value is used to reflect the accuracy of the content of the candidate dialogue text. Based on the comprehensive reward value, a target dialogue text is selected from the candidate dialogue texts for use in a dialogue with the user.

2. The method as described in claim 1, wherein each candidate dialogue text is evaluated to determine a first reward value corresponding to each candidate dialogue text, specifically including: For each candidate dialogue text, feature extraction is performed on the candidate dialogue text to obtain the text feature representation corresponding to the candidate dialogue text; as well as Feature extraction is performed on the dialogue state data to obtain the dialogue feature representation of the current dialogue; The first reward value corresponding to the candidate dialogue text is determined based on the similarity between the text feature representation and the dialogue feature representation.

3. The method as described in claim 2, wherein for each candidate dialogue text, feature extraction is performed on the candidate dialogue text to obtain the text feature representation corresponding to the candidate dialogue text, specifically including: For each candidate dialogue text, the candidate dialogue text is input into the dialogue text encoding tower of the pre-set prediction response reward model to obtain the text feature representation corresponding to the candidate dialogue text; Feature extraction is performed on the dialogue state data to obtain the dialogue feature representation of the current dialogue, specifically including: The dialogue state data is input into the state data encoding tower of the predicted response reward model to obtain the dialogue feature representation of the current dialogue.

4. The method as described in claim 2, wherein a first reward value corresponding to the candidate dialogue text is determined based on the similarity between the text feature representation and the dialogue feature representation, specifically including: The user's estimated response rate is determined based on the similarity between the text feature representation and the dialogue feature representation; wherein the estimated response rate is positively correlated with the similarity. Based on the estimated response rate, determine the first reward value corresponding to the candidate dialogue text.

5. The method of claim 1, wherein each candidate dialogue text is evaluated to determine a second reward value corresponding to each candidate dialogue text, specifically comprising: For each candidate dialogue text, generate the corresponding prompt text based on that candidate dialogue text; The prompt text is used to guide the preset accuracy reward model to determine the second reward value corresponding to the candidate dialogue text according to the preset evaluation dimensions; The prompt text is input into the accuracy reward model to obtain the second reward value corresponding to the candidate dialogue text.

6. The method of claim 5, wherein the evaluation dimensions include: The compliance dimension is used to characterize the satisfaction rate of the candidate dialogue text with each preset expression goal; Authenticity dimension used to characterize the authenticity of information in the candidate dialogue text; At least one of the following: a compliance dimension used to characterize whether the candidate dialogue text conforms to the preset risk control rules; and an expressive dimension used to characterize the linguistic fluency of the candidate dialogue text.

7. The method as described in claim 1, wherein the first reward value and the second reward value are combined to obtain a comprehensive reward value, specifically includes: For each candidate dialogue text, an initial comprehensive reward value is determined based on the first reward value and the second reward value. The overall reward value of a candidate dialogue text is determined based on the mean and / or standard deviation of the initial overall reward values ​​of each candidate dialogue text; the overall reward value is used to reflect the degree of advantage of the initial overall reward value of the candidate dialogue text relative to the initial overall reward values ​​of other candidate dialogue texts.

8. The method of claim 1, wherein the generative model is trained in the following manner: Input the preset sample dialogue state data into the preset initial generation model to generate candidate dialogue text for each sample. Determine the comprehensive reward value corresponding to each sample candidate dialogue text; Based on the comprehensive reward value corresponding to each sample candidate dialogue text, the sample target dialogue text is selected from each sample candidate dialogue text; The target loss is determined based on the difference between the comprehensive reward value of the target dialogue text and the comprehensive reward value of other candidate dialogue texts; the target loss is positively correlated with the difference. The initial generative model is trained with the goal of minimizing the target loss as the optimization objective, and the generative model is obtained.

9. A dialogue text generation device, comprising: The acquisition module is used to acquire the dialogue state data of the current conversation; The dialogue status data includes historical dialogue records between the customer service representative and the user involved in the current dialogue, as well as the user's attribute data; The generation module is used to input the dialogue state data into a preset generation model to obtain candidate dialogue texts. The determination module is used to evaluate each candidate dialogue text to determine a first reward value and a second reward value corresponding to each candidate dialogue text, and to merge the first reward value and the second reward value to obtain a comprehensive reward value; the first reward value is used to reflect the estimated response rate of the user after receiving the candidate dialogue text; the second reward value is used to reflect the professionalism and compliance of the candidate dialogue text. The selection module is used to select a target dialogue text from the candidate dialogue texts based on the comprehensive reward value, for use in dialogue with the user.

10. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-8 by executing the executable instructions.

11. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-8.

12. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-8.

Images