Intelligent customer service session processing method and device and program product

Abstract

The embodiment of the invention relates to the field of large models, and provides an intelligent customer service session processing method and device and a program product, which are applied to an e-commerce customer service system based on a multi-modal large model, and the method comprises the following steps: obtaining a first input of a target user on a first session interface; obtaining a first key feature associated with a resource in the e-commerce transaction platform from the first input; performing content enhancement on the first key feature based on a generative model to obtain a target search field set, the target search field set comprising a plurality of search fields having semantic association with the first key feature; obtaining a target answer set, wherein the target answer set comprises candidate answers matched with the search fields; and sending and displaying the target answer set on the first session interface according to a preset strategy. According to the scheme, enhancement can be performed by using the query generation model based on the user intention, and the accuracy of identifying the user intention is improved through dual mechanisms, so that the session efficiency and the session quality are improved, and the operation cost of a customer service system is saved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of large model technology, and in particular to an intelligent customer service conversation processing method, device and program product. Background Art

[0002] In the field of e-commerce customer service, customer service robots have become an indispensable tool. Customer service robots will automatically answer users' questions, which can improve customer service efficiency, respond to users in a timely manner, and improve user experience.

[0003] In the process of research and practice of the prior art, the inventors of the embodiments of the present application found that, at present, the existing customer service robots are mainly based on text queries and do not truly understand the user's intentions. The e-commerce trading platform designs a wide variety of businesses, and the business data is complicated and diversified. When facing users of different product types (including merchants and buyers), various inquiries that do not comply with the preset machine rules will appear, which ultimately leads to the customer service robots' inaccurate answers, inability to understand the user's intentions, and inability to handle complex problems. Summary of the invention

[0004] The embodiments of the present application provide a method, device and program product for processing intelligent customer service conversations. By implanting a multimodal intelligent customer service big model into a business server, the intelligent customer service big model can initially identify the user's intent based on the user's query, and then use the query generation model to enhance it based on the user's intent, thereby improving the accuracy of identifying the user's intent through a dual mechanism, thereby improving conversation efficiency and conversation quality, and saving the operating costs of the customer service system.

[0005] In a first aspect, an embodiment of the present application provides an intelligent customer service session processing method. The method is applied to an e-commerce customer service system based on a multimodal large model, and the method includes: Obtaining a first input of a target user in a first conversation interface; Acquire a first key feature associated with a resource in the e-commerce transaction platform from the first input; Performing content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes a plurality of search fields that are semantically associated with the first key feature; Acquire a target answer set, the target answer set including candidate answers matching each search field in the target search field set; According to a preset strategy, the target answer set is sent and displayed on the first conversation interface.

[0006] In a second aspect, an embodiment of the present application provides a conversation processing device having the function of implementing the intelligent customer service conversation processing method provided in the first aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0007] In some implementations, the session processing device includes: A display module, used to display a first conversation interface between the intelligent customer service model and the target user; An input-output module, used to obtain a first input of a target user in a first conversation interface; A processing module is used to obtain a first input of a target user in a first conversation interface; obtain a first key feature associated with a resource in an e-commerce transaction platform from the first input; perform content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes multiple search fields that are semantically associated with the first key feature; obtain a target answer set, wherein the target answer set includes candidate answers that match each search field in the target search field set; and according to a preset strategy, send the target answer set and display it on the first conversation interface through the display module.

[0008] In some implementations, the processing module is specifically used to: An initial answer set is obtained, wherein the initial answer set includes candidate answers that match each search field in the target search field set; a priori knowledge base or a third-party trusted platform is called to calculate the confidence of each answer in the initial answer set; candidate answers whose confidence is lower than a preset confidence are removed from the initial answer set to obtain the target answer set.

[0009] In some embodiments, the first input is a query for historical orders within a first historical period, and the target answer set includes a first candidate answer for the order list of the target user within the first historical period; the processing module is also used to: traverse the e-commerce transaction system; if it is verified that the order information in the candidate answers does not belong to a real transaction order in the e-commerce transaction system, the candidate answers that do not belong to the real transaction order are removed from the initial answer set to obtain the target answer set; and the first answer including the target answer set is displayed on the first session interface through the display module.

[0010] In some embodiments, the first input is a query or recommendation of resources to be purchased by a target user within a second historical period, and the target answer set includes a second candidate answer for the target user's list of resources to be purchased within the second historical period; the processing module is also used to: determine the resource type and purchase intention based on the first key feature; obtain candidate resources to be purchased that match the resource type from the target user's shopping cart; determine a list of resources to be purchased from the candidate resources to be purchased based on the target user's shopping habits and purchase intentions; and display the second candidate answer including the list of resources to be purchased in the first conversation interface through the display module.

[0011] In some embodiments, the first input is to query the after-sales data of the target user within a third historical period, and the target answer set includes a third candidate answer for the after-sales list of the target user within the third historical period; the processing module is also used to: determine the after-sales resource type and after-sales type according to the first key feature; obtain candidate after-sales data matching the after-sales resource type and after-sales type from the shopping cart of the target user; determine the after-sales list from the candidate after-sales data according to the after-sales quantity and after-sales status of the resources; display the third candidate answer including the after-sales list in the first conversation interface through the display module; the third candidate answer is a first visualization chart, and the visualization chart includes a dynamic switching icon; the first visualization chart is obtained based on the after-sales order, after-sales time, after-sales category, after-sales status, after-sales result and after-sales personnel.

[0012] In some embodiments, the first input is to query the target user's transaction data on distribution resources on different e-commerce trading platforms within a fourth historical period, and the target answer set includes a fourth candidate answer of the target user's list of distributed resources within the fourth historical period; the processing module is also used to: obtain the first transaction data of the target user's distributed resources on different e-commerce trading platforms through the input-output module; determine first description information of the resources to be distributed based on the first transaction data, the first description information including the information of the resources to be distributed and the e-commerce trading platform to be distributed; determine the resource distribution strategy based on the first description information; display the fourth candidate answer including the resource distribution strategy in the first conversation interface; the fourth candidate answer is a second visualization chart, and the second visualization chart includes the smart distribution application icon, and the information of the resources to be distributed on different e-commerce trading platforms.

[0013] In some implementations, the first input is to query the financial data of the target user on different e-commerce trading platforms in a fifth historical period, and the target answer set includes a fifth candidate answer of a financial list of historical transaction resources of the target user in the fifth historical period; the processing module is further used to: The input-output module is used to obtain second transaction data of resource transactions of target users on different e-commerce transaction platforms; the financial data of historical transaction resources are determined based on the second transaction data; financial statements are generated based on the financial data; and a fifth candidate answer including the financial statement is displayed in the first conversation interface through the display module; the fifth candidate answer is a third visualization chart, and the third visualization chart includes a dynamic switching icon; the third visualization chart is obtained based on historical transaction orders, settlement data and cash flow.

[0014] In some embodiments, the e-commerce customer service system also includes an adversarial generation model and a query generation model, and the processing module is also used to: obtain a first prompt word input into the adversarial generation model through the input-output module; generate a first question set of the e-commerce transaction platform based on the first prompt word, the first question set includes multiple adversarial e-commerce questions; input the first question set into the query generation model to perform adversarial training on the query generation model.

[0015] In a third aspect, an embodiment of the present application provides a session processing device, comprising: at least one processor and a memory; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the above-mentioned first aspect, any one implementation method of the first aspect, or any step of the second aspect to implement the intelligent customer service session processing method.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a function of implementing the intelligent customer service session processing method provided in the first aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. Specifically, the computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for the processor to load to execute any step of implementing the intelligent customer service session processing method provided in the first aspect or the second aspect of the embodiment of the present application.

[0017] Compared with the prior art, in the solution provided by the embodiment of the present application, since the intelligent customer service big model is based on multimodality, it can understand any modal data input by the user. Therefore, on the basis of preliminarily identifying the user's intention based on the first key feature in the first input, the generative model is used to enhance the content based on the user's intention to obtain a plurality of search fields. Since these search fields are all semantically associated with the first key feature, on the one hand, the plurality of search fields can further determine the user's intention, so that the target answer set obtained based on the plurality of search fields can better match the user's intention. On the other hand, when the intelligent customer service big model and the user have a real-time conversation, the user does not need to change the content of the input query multiple times to express his or her demands, which significantly reduces the user's inquiry + answer acquisition time, and significantly improves the quality inspection efficiency and the reception quality of the intelligent customer service big model, saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1a A schematic diagram of a scenario of a communication system for implementing an intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 1b A schematic diagram of a scenario of a communication system for implementing an intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 2a A flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 2b A schematic diagram of an interface for processing after-sales inquiries provided in an embodiment of the present application; Figure 2c A schematic diagram of query data enhancement performed by the query generation model provided in this application; Figure 2d A schematic diagram of a framework for adversarial training of a query generation model provided in this application; Figure 2e A schematic diagram of query data enhancement performed by the query generation model provided in this application; Figure 2f A schematic diagram of query data enhancement by the query generation model provided in this application; Figure 2g A schematic diagram of a scenario for verifying an answer provided in an embodiment of the present application; Figure 3aA flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 3b A schematic diagram of an interface for verifying order query provided in an embodiment of the present application; Figure 4 A flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 5a A flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 5b A schematic diagram of an interface for responding to after-sales list output provided in an embodiment of the present application; Figure 6a A flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 6b A schematic diagram of an interface for responding to the output of a product to be distributed provided in an embodiment of the present application; Figure 7 A flowchart of the intelligent customer service conversation processing method provided in an embodiment of the present application; Figure 8 A schematic diagram of an interface for responding to financial data output provided in an embodiment of the present application; Fig. 9 A schematic diagram of the structure of a session processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solution of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. The described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0020] The terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects (for example, the first input and the second input in the embodiments of the present application respectively represent different messages of the user input conversation interface), and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In some of the processes described in the embodiments of the present application and the drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in the embodiments of the present application or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in the embodiments of the present application is only a logical division. There may be other division methods when implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0021] The embodiments of the present application provide a method, apparatus and program product for processing intelligent customer service conversations, which can be used in servers or terminal devices. Specifically, it can be used in e-commerce trading platforms, social platforms, financial platforms, government platforms, retrieval systems and other platforms that need to introduce AI intelligent customer service to conduct efficient conversations with users. For example, it can be applied to a customer service system implanted in a business server. The intelligent customer service big model can conduct efficient conversations with users offline or online in real time, thereby accurately understanding the content of the conversation, not limited to making judgments based on the preset rules of the e-commerce trading platform, timely avoiding or reducing misjudgments, and significantly improving conversation efficiency and customer service reception quality, saving labor costs.

[0022] In some implementations, the present invention is applied to Figure 1aThe communication system shown in the figure may include at least one business server 1, a customer service system 2, and at least one user terminal 3 (only one user terminal is shown in Figure 1). Among them, the customer service system 2 includes a query generation model and at least one intelligent customer service large model. The user terminal 3 can be a server or a terminal. When it is a terminal, it has a client installed. The client installed on each user terminal can be a client with the same or different version numbers, or a client with the same or different operating systems. The embodiment of the present application does not limit the version of the client and the type of operating system corresponding to the client. The following is a detailed description.

[0023] Business server 1: can be used to provide business services such as search, transaction, conversation, upload, download, etc. to users of each user terminal. For example, the server can provide users with resource transactions (such as purchasing things on an e-commerce transaction platform), connect to customer service for real-time conversation, etc. The business server can be an e-commerce transaction platform, a social platform, a financial platform, a government platform, a retrieval system, and other platforms that need to implement semantic inquiries. You can refer to the embodiments of this application regarding the e-commerce transaction platform, and will not repeat them here. This application does not limit this.

[0024] Customer service system 2: The customer service system 2 includes a query generation model and at least one intelligent customer service big model. The query generation model can be used to perform data enhancement on the initial query received by the intelligent customer service big model from the user, obtain multiple candidate queries, and feed them back to the intelligent customer service big model. It can be used to answer the input of user a of user terminal 3 by the intelligent customer service big model.

[0025] In some implementations, the customer service system 2 includes a large quality inspection model (not shown in the figure). Optionally, the customer service system 2 may also include an adversarial generation model (such as Figure 1b The customer service system 2 and the intelligent customer service big model can both be run in the business server in the form of plug-ins, for example, in the e-commerce trading platform in the form of plug-ins, to perform quality inspection on the answer of the intelligent customer service big model to user a.

[0026] Among them, the adversarial generation model has two main functions in this application: generating multiple adversarial e-commerce questions based on prompt words, which are used to perform adversarial training on the intelligent customer service model to improve the robustness of the intelligent customer service model; and performing adversarial training on the query generation of the query generation model to obtain candidate queries that are more natural and better match user intentions.

[0027] The quality inspection big model is used to score the answers fed back by the intelligent customer service big model based on the adversarial questions continuously input by the adversarial generation model. The quality inspection big model can be used only in a specific e-commerce transaction field, or it can cover any application field that is set up based on a multimodal big model to implement conversations with users. It can be selected based on the actual application direction of the intelligent customer service big model, and this application does not limit this. The quality inspection big model covers a variety of conversation scenarios and involves multiple evaluation dimensions. The specific conversation scenarios can refer to the various processing strategies, e-commerce platform regulations, and relevant laws and regulations in the e-commerce field for user consultation, pre-sale of resources, and after-sale of resources, etc., which are not limited in this application. The quality inspection big model, teacher big model, adversarial generation model, and intelligent customer service big model involved in the embodiments of this application are all large language models, which can specifically be multimodal large language models (MLLMs) based on LLM evolution. Since the quality inspection big model, teacher big model, adversarial generation model and intelligent customer service big model involved are all large language models, they bridge the gap between visual data and language context, enabling these big models to understand and generate content that combines text, images and other modalities, and are therefore more conducive to the quality evaluation of the intelligent customer service big model in this application.

[0028] User terminal 3: Install a client that matches the server deployed in the business server, and use the client to communicate with the intelligent customer service model in the customer service system. The user terminal of this application is equivalent to the terminal device and is not distinguished.

[0029] When based on Figure 1a or Figure 1b When the communication system shown implements the above-mentioned intelligent customer service conversation processing method, the customer service evaluation system running in the business server in a plug-in manner processes the conversation between the intelligent customer service big model and the user offline or online in real time, thereby achieving accurate understanding of the conversation content. It is not limited to responding according to the preset rules of the e-commerce transaction platform, significantly improving conversation efficiency and customer service reception quality, and saving labor costs.

[0030] It can be understood that the customer service system in the present application can be applied to the service side of the e-commerce trading platform (i.e., processing the conversation between the official customer service of the e-commerce trading platform and the user), and can also be used for the store service of a single merchant (i.e., processing the conversation between the store customer service and the user for quality inspection). This application mainly takes the conversation processing between the official customer service of the e-commerce trading platform and the user as an example. Other scenarios can be used as a reference and will not be elaborated on.

[0031] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence (AI), natural language processing (NLP), and machine learning (ML).

[0032] Regarding the conversation processing direction of the intelligent customer service big model in the field of artificial intelligence, this application can implant a multimodal intelligent customer service big model into the business server. The intelligent customer service big model can initially identify the user's intention based on the user's query, and then use the query generation model to enhance it based on the user's intention, thereby improving the accuracy of identifying the user's intention through a dual mechanism, thereby improving the conversation efficiency and conversation quality.

[0033] It should be noted that the server (such as a business server, a search engine) involved in the embodiments of the present application may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. When the computing device in the embodiments of the present application is a terminal device, the embodiments of the present application also provide a terminal device, which may be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc.

[0034] In order to solve the problems of low accuracy of user intent recognition and low conversation quality in related technologies, the embodiments of the present application mainly adopt the following technical solutions: 1. Accurately identify user intent: Introduce a large model of intelligent customer service based on multimodality, preliminarily identify user intent based on the initial query, and then use the query generation model based on user intent to strengthen the input, obtain multiple candidate queries, and further determine user intent. Specifically, the customer service system running in the business server as a plug-in can conduct efficient conversations with users offline or online in real time. Users do not need to change the content of the input query multiple times to express their demands, which significantly reduces the time it takes for users to inquire and get answers, and significantly improves quality inspection efficiency and customer service reception quality, saving labor costs.

[0035] 2. Deploy multiple intelligent customer service big models: For the intelligent customer service big model deployed on the e-commerce transaction platform, since the training set of the intelligent customer service big model can be trained based on part or all of the business data and user data, the business type that the intelligent customer service big model can respond to can also be a dedicated or public intelligent customer service big model covering all business types.

[0036] 3. Improve the naturalness and accuracy of queries: By generating adversarial models to conduct adversarial training on the query generation model, it is possible to ensure that the generated candidate queries are more natural and diverse, while also improving the effectiveness and accuracy of candidate queries in e-commerce transaction query scenarios.

[0037] The following combination Figure 2a-Figure 9 The technical solution of the embodiment of the present application is exemplified.

[0038] Since the intelligent customer service conversation processing method of the embodiment of the present application processes the output (i.e., the answer to the question input by the user) of a pre-trained intelligent customer service big model, the training process of the intelligent customer service big model is first introduced before introducing the intelligent customer service conversation processing method. The training process includes: 1. Data Level 1. Data Collection 1.1 Historical customer service session data Collect historical chat records stored in the customer service system, including user questions, customer service responses, and corresponding product information (such as product name, price, specifications, event information, etc.). Extract the complete conversation context (including multiple rounds of conversations) and mark the conversations that can be clearly linked to the product or user ordering behavior (such as "check specifications", "add to shopping cart", "ask about delivery time", etc.).

[0039] 1.2 Product Knowledge Base / Product Information Database Contains rich information such as product name, category, brand, specification, price, inventory status, discount information, logistics rules, etc. This data can be stored in a database or knowledge graph for real-time query in conversations.

[0040] 1.3. FAQ Question and Answer Data Summarize frequently asked questions (such as "how to return goods", "how to use coupons", "invoice issuance methods", etc.), including questions and standard answers. This part of the data can be used as part of the knowledge base and will also be combined with customer service conversation data.

[0041] 1.4. Customer behavior data (optional) If more refined user personalized services are needed, user behavior data can be integrated, including user browsing history, favorites, add-to-cart, purchase history, evaluation feedback, etc. This type of data is useful for personalized scenarios, such as multi-round related conversations such as "Have you restocked the item you bought last time?"

[0042] Pay attention to privacy and compliance: When collecting and using business data, you need to comply with relevant privacy policies and regulations, anonymize and protect sensitive information.

[0043] 2. Data cleaning and labeling 2.12 Data deduplication and noise filtering De-duplicate the collected historical conversation data and filter out conversations that are irrelevant to the business or have extremely high noise (such as obvious idle chats, insults, and test texts).

[0044] 2.2 Dialogue segmentation and structuring Divide the conversation into segments or rounds based on each user question and customer service response. Make sure you can build a multi-round data format, such as: { "role": "user", "content": "Hello, is this shoe available in black?" }, { "role": "assistant", "content": "Hello, this shoe is currently available in black and white. Which one do you want?" } At the same time, the corresponding product information is extracted, or the inventory, specifications and other information queried by the customer service in the system.

[0045] 3. Business intent and slot labeling (optional) In order to enable the model to better understand user intent and extract key information (such as product name, color, size, payment method, etc.), historical conversations can be classified and slots can be marked: Intent tags: inquiring about inventory, asking about prices, asking about delivery time, return and exchange consultation, coupon information, etc. Slots: product name, color, size, user address, return reason, etc.

[0046] 4. FAQ alignment and supplement Normalize the questions and answers in the FAQ to keep the content clear and the format consistent. For example, complete some incomplete questions or format the answers into paragraphs or lists.

[0047] 2. Model training and verification 1. Model selection and strategy 1.1 Pre-trained language model + custom fine-tuning The commonly used approach at present is to first select a large-scale pre-trained language model (such as a GPT model), and then fine-tune it based on e-commerce data.

[0048] 1.2 Combining search and generation When the questions raised by users clearly require business data support (such as dynamic data such as inventory, price, order status, etc.), the customer service system first searches the database or knowledge base to obtain the answer, and then integrates the search results into the model reply. For fixed questions such as FAQs and general questions and answers, the search mode can be used to quickly provide templates or standard answers. For complex or open-ended questions, a generative model is used to understand and generate answers.

[0049] 2. Strengthen the connection between models and business data 2.1 Knowledge base / database real-time query Before generating an answer, the big model first searches and queries the knowledge base / database for specific intents or keywords (such as "Is this product in stock?", "What is its material?", "What is the promotional price?").

[0050] Return structured information (such as inventory = 123, price = 199 yuan) to the model as context or prompt for the generative model to use. This allows the model to provide the latest and most accurate business data in its answers.

[0051] 2.2 Prompt Engineering / Model Interface Design When users interact with the intelligent customer service model, business rules, data query results, product information, etc. are injected into the model context through prompts. For example: "You are now a customer service representative of a certain e-commerce platform. Please answer the following questions based on the product information and the user's question. The product information includes: name =..., price =..., inventory =..., and the following is the user's question:..." By injecting policies into prompts or system information, the model can prioritize the use of the provided business data for answers and limit the answer format (for example, requiring the provision of links / specification information, etc.).

[0052] 2.3 Fine-tune the data to highlight key business entities In the data of fine-tuning, a large number of real "user questions + query results + final customer service answers" triples can be placed. Through fine-tuning, the large model can learn: when the user's questions contain certain keywords, how to use the corresponding business information to answer.

[0053] 3. Training Process 3.1 Data Partitioning The complete corpus is divided into training set, validation set and test set. The typical division method can be 8:1:1 or 7:2:1, etc., to ensure generalization ability.

[0054] 3.2 Fine-tuning stage When based on a generative large language model (LLM), command fine-tuning or dialogue fine-tuning is usually done first: (1) Instruction Tuning: Collect common instructions and their correct responses in e-commerce customer service scenarios, such as “list five recommended products and give prices”. (2) Conversation Tuning: Historical conversation data is input into the model as multiple rounds of samples, and the model is trained to give appropriate responses based on the user context.

[0055] At the same time, examples of FAQ questions and product information query scenarios can also be added to the training data to allow the model to learn standard answer formats.

[0056] (3) Model evaluation indicators Accuracy: whether the key business information (price, inventory, etc.) involved in the answer is accurate; Semantic matching: the matching degree between the answer and the user's question, and whether the answer is irrelevant; Manual scoring: Manual evaluation of the generated results, including conversation fluency, politeness, whether it crosses sensitive lines or violates prohibitions, etc. Business success rate: such as completing transactions, solving problems, improving customer service satisfaction, reducing secondary interactions, etc. These can be counted during the online trial run.

[0057] 4. Verification and testing 4.1 Offline testing Use the validation set and test set to evaluate the model offline and measure dialogue generation indicators such as accuracy, recall, semantic matching, and BLEU.

[0058] 4.2 Online A / B Testing Small-scale launch to real customer service scenarios, conduct A / B testing, and compare the differences between the new model and the old customer service system in key indicators such as user satisfaction and problem resolution rate. You can also do a grayscale release to open the new model service to a part of users or a part of functional scenarios first.

[0059] 4.3 Model quality inspection Several cases are extracted from real conversations and professional customer service or business experts are asked to conduct quality inspection and scoring to identify deficiencies and errors in the model and make iterative corrections.

[0060] As described in the above embodiment, after the above intelligent customer service big model is obtained through model training, the user's intention can be accurately understood based on the intelligent customer service big model, and the answer matching the user's intention can be output to the user's inquiry to improve the conversation quality between the intelligent customer service big model and the user, so as to respond to the user in a timely manner, handle the user's problems and other business needs. Specifically, see Figure 2a A method for processing intelligent customer service conversations is provided, which is applied to a customer service system based on a multimodal large model, and is implemented based on supervised learning LLM. The method can be executed by a business server or a terminal device, and the business server can be an e-commerce trading platform, a social platform, a financial platform, a government affairs platform, a retrieval system, or other platform that needs to process conversations between customer service and users.

[0061] Among them, the terminal device is a device installed with an e-commerce transaction platform client. The device implementing the above-mentioned intelligent customer service session processing method can be run in the e-commerce transaction platform client and the e-commerce transaction platform server in a plug-in manner, or it can be directly integrated and designed in the e-commerce transaction platform client and the e-commerce transaction platform server. The specific operation mode is not limited in this embodiment of the application. The subsequent embodiments only take the server and the e-commerce transaction platform server applied to the e-commerce transaction platform as examples. The embodiments of the present application include: 201. Obtain a first input of a target user in a first conversation interface.

[0062] The first conversation interface is a conversation interface between the intelligent customer service model and the target user. The first conversation interface refers to the display interface of the conversation (which can be a real-time conversation or a historical conversation) established between the user and the intelligent customer service model in the e-commerce transaction platform when the user logs into the client of the e-commerce transaction platform. Both the target user and the intelligent customer service model can send and receive messages of any mode in the target conversation interface. The first input includes at least two messages of at least one mode.

[0063] It can be understood that the first input may include multimodal media data such as text, pictures, voice, video, expressions (dynamic or static), etc. The intelligent customer service big model in the embodiment of the present application is trained based on multimodal data, so it can understand data of any modality in the conversation, and can also send data of any modality in the conversation interface that matches user intentions or solves user demands. The embodiment of the present application takes the target user inputting text and images in the conversation interface as an example. When the target user inputs messages of other modalities, the intelligent customer service big model can analyze, integrate and other operations on messages of any modality for easy understanding, without any limitation on this.

[0064] In some implementations, for the above-mentioned first input, the first input includes many messages sent by users. If the intelligent customer service model is to quickly give an answer that matches the user's intention within a certain period of time, it is necessary to first infer the contextual relationship between each message in the first input.

[0065] The contextual relationship of each message can be realized based on in-context learning. The quality inspection model is a large language model (LLM). When training, it can be based on pre-acquired training data (including various task descriptions-corresponding answer pair structure data). The attention layers in the LLM can realize contextual reasoning. For example, the first user is asking about how to use the function of the e-commerce trading platform's delivery assistant. The intelligent customer service model quickly understands the user's intention of the first user and gives the corresponding operation strategy to ensure smooth communication throughout the conversation.

[0066] The emotional data of the target user may be reflected in the above-mentioned first input, and the emotional data can also be realized based on the high-order perception ability of LLM, that is, by extracting the deep meaning of the messages of each modality in the first input of the target user and the answer of the intelligent customer service big model, and deriving the implicit meaning from the visual and contextual relationship, so as to realize the emotional perception of the target user in the target time period. This will help the intelligent customer service big model to preliminarily determine the user's intention and user emotion. It is understandable that when the intelligent customer service big model extracts and understands the features of each modal information in the first input, if the emotional data is perceived, the emotional data also needs to be added to the first key feature described below.

[0067] 202. Obtain a first key feature associated with resources in the e-commerce transaction platform from the first input, wherein the resources in the e-commerce transaction platform refer to various tradable resources provided to users, including but not limited to virtual resources, various commodities, etc.

[0068] The first key feature may include descriptive information extracted from any modal data associated with any resource currently consulted by the user, and may also include emotional features perceived based on these modal data. The first key feature including emotional features is more helpful for the intelligent customer service model to clarify the user's intention.

[0069] The first key feature refers to the content feature that can describe the various modal data in the first input. For example, if the user sends a combination of at least one modality such as text, image, audio, and video in the first conversation interface, the intelligent customer service model can select the corresponding feature extraction method based on the modality of the data in the first input to obtain the content feature that can describe the various modal data in the first input from the first input.

[0070] For example, if a user inputs an image, the image description information of the image can be extracted based on recurrent neural networks and convolutional neural networks. The image description information is a piece of text that describes the image content in natural language, aiming to accurately convey the visual information in the image through text, including but not limited to the composition of the scene, the existence of objects and their attributes (such as color, size), the relationship between objects, and possible actions or events. After extracting the image description information, the intelligent customer service model can preliminarily determine the user's intention for this input.

[0071] like Figure 2b As shown, the image sent by the user is an image of a damaged rice cooker. Then the intelligent customer service model can search for the target order of the rice cooker in the orders recently purchased by the user, and forward the target order to the conversation interface, along with a text or image response to confirm the specific damage of the rice cooker. It can be seen that the user does not need to make a phone call, type a word, or even search one by one from "My Orders" and then forward it to the conversation interface. The user's demands can be quickly captured by the intelligent customer service model, and the corresponding processing strategy can be directly given. Obviously, this solution can significantly improve conversation efficiency and conversation effect.

[0072] Optionally, in some embodiments, backdoor implants and triggers can also be detected to avoid backdoor attacks, or cause the large model to misidentify the user's intention, ultimately resulting in the output of a specified result that does not meet the user's intention. Specifically, before obtaining the first key feature, it is also possible to detect whether the first input contains noise that meets the preset conditions, and the preset conditions include at least one of triggers and grammatical errors; if included, the noise in the first input is removed. It can be seen that by pre-detecting and filtering the noise in the first input that meets the first preset condition, it is possible to avoid the intelligent customer service large model from outputting an incorrect answer or misidentifying the user's first input, thereby improving the output quality.

[0073] 203. Perform content enhancement on the first key feature based on a generative model to obtain a target search field set.

[0074] The target search field set includes multiple search fields that are semantically associated with the first key feature. For example, when the first input of the target user is for querying order information of an order system, each search field in the target search field set can be used to search for order information in the order system.

[0075] In some implementations, the generative model can be a query generation model. In an e-commerce trading platform, when faced with an initial query input by a user, a query generation model can be used to enhance the input based on the user intent identified by the initial query, and multiple candidate queries (which may include the initial query) can be obtained. Then, a deep search is performed based on the answers that match the multiple candidate queries, and finally the answers to all candidate queries are integrated, thereby improving the accuracy of identifying the user intent.

[0076] For example Figure 2c As shown, the initial query entered by the user is: Why is there still no product in the store? The initial query is enhanced based on the query generation model to obtain the following candidate queries: Are there any products in my store? Did the distribution fail? Why was the product not successfully listed? … Based on the above multiple candidate queries, multi-intent queries can be performed to further determine the user's intent, especially when the initial query entered by the user is not comprehensive. On the one hand, it can intelligently help the user expand multiple more detailed and comprehensive candidate queries. On the other hand, it can reduce the need for users to change the content of the input query multiple times to express their demands, and accordingly significantly reduce the user's inquiry + answer time, thereby improving the user experience.

[0077] It is understandable that in some implementations, the query generation model in the embodiments of the present application can be trained adversarially through a generative adversarial network (GANs) to make the generated candidate queries more natural and diverse, such as Figure 2d A frame design is shown.

[0078] The query generation model can be trained in conjunction with query tasks related to e-commerce transactions in an e-commerce transaction platform, so that the query generation model can simultaneously understand and query information related to e-commerce transactions, so as to correctly understand user intentions and output credible and controllable answers to users.

[0079] Optionally, in some embodiments of the present application, such as Figure 2d As shown, the e-commerce customer service system also includes an adversarial generation model and a query generation model, and the quality of candidate query generation generated by the query generation model is improved by performing adversarial training on the query generation model. Specifically, the method also includes: S1, obtaining a first prompt word input into the adversarial generation model; S2. Generate a first set of questions for the e-commerce trading platform based on the first prompt word, wherein the first set of questions includes a plurality of adversarial e-commerce questions; S3. Input the first question set into the query generation model to perform adversarial training on the query generation model.

[0080] For example, S3 includes the following steps: S31: Acquire multi-dimensional business data in the field of e-commerce transactions, and query samples generated by the generator based on input of preset tasks.

[0081] The business data includes business data of various data types, such as transaction data, product data, inventory data, marketing data, logistics data, after-sales data and financial data. The business data is a real sample of e-commerce transaction-related data.

[0082] S32: Train a discriminator based on the business data and query samples obtained in step 1, so that the discriminator distinguishes between real samples and query samples, that is, maximizes the edit distance between the real business data and the query samples based on the first loss function.

[0083] S33: Minimize the edit distance between the real business data and the query sample based on the second loss function, so that the query sample generated by the generator is misclassified as a real sample.

[0084] For example, let’s take the following five key tasks as examples to explain how to perform data augmentation based on the query generation model: Task 1: Product Search Input: User natural language. For example, "I want to buy a pair of running shoes" or "Nike's latest running shoes, budget less than 300 yuan".

[0085] Output: structured or semi-structured retrieval query.

[0086] Training Data: Historical search logs (such as the user's original query, the query / filter conditions used by the final retrieval system, click / purchase feedback), product category system, product attribute information (which can be a product knowledge graph) like Figure 2e As shown in the figure, the user query is: Nike's latest running shoes, budget less than 300 yuan. After the query generation model enhances the user query: Search Query: {Brand: "Nike", Product Type: "Running Shoes", Maximum Price: 300}) or other executable search statements.

[0087] Task 2: Order Query Input: Natural language, such as "I want to check the order of the mobile phone I bought yesterday", or "Please help me find out where the white shirt I bought last month is?" Output: Parameters required by the query order service (such as: {user ID: 12345, query type: "logistics status", product: "white shirt", time range: "last month"}).

[0088] Training Data: User dialogue / customer service system log: (user questions, actual order service interface and parameters called, order return results / dialogue replies) like Figure 2e As shown in the figure, the user query is: Please help me check where the white shirt I bought last month is? After the query generation model enhances the user query: Search Query: {User ID: 12345, Query type: "Logistics status", Product: "White shirt", Time range: "Last month"} Task 3: After-sales / customer service questions and answers Input: Natural language, such as "What should I do if this dress doesn't fit and I want to return it?" or "How do I issue an electronic invoice?"

[0089] Output: 1) A natural language answer given to the user; 2) It may be necessary to call the after-sales service interface internally ({after-sales type: "return", order number: xxxxx}).

[0090] Training Data: Customer service knowledge base (frequently asked questions and answers), historical customer service conversation records: (user questions, final answers given by manual customer service or FAQ system), after-sales service call log like Figure 2e As shown, the user query is: What should I do if this dress doesn’t fit and I want to return it? After the query generation model enhances the user query: Search Query: {After-sales type: "return", order number: xxxxx}) Task 4: Product attribute questions and answers Input: Natural language, such as "Do you have size 41 for this shoe?" or "Can I install Windows 10 on this computer?" Output: natural language answer + internal search query (for example, query inventory and attributes in the product details database).

[0091] Training Data: Product knowledge base: product information (attributes, specifications, inventory), historical question and answer log: (user questions, structured queries, and answers presented to users) Task 5: Multi-turn dialogue management Input: contextual multi-turn dialogue, for example: 1. “I want to buy a mobile phone” 2. "Budget is around 2,000" 3. “Huawei is the best” Output: updated search intent or dialog query (filter conditions of accumulated context: {product type: "mobile phone", brand: "Huawei", price range: [1500, 2000]}).

[0092] Training Data: Multi-round conversation logs, multi-round refinement traces in search systems or customer service systems For example Figure 2f As shown in the figure, the user query is: "I want to buy a mobile phone", "budget is about 2,000", "preferably Huawei". After the query generation model enhances the user query: Search Query: {Product type: "Mobile phone", Brand: "Huawei", Price range: [1500, 2000]} It can be seen that by performing adversarial training on the query generation model through the generative adversarial model, it is possible to ensure that the generated candidate queries are more natural and diverse, while also improving the effectiveness and accuracy of the candidate queries in the e-commerce transaction query scenario.

[0093] 204. Obtain a target answer set, wherein the target answer set includes candidate answers that match each search field in the target search field set.

[0094] In some implementations, the target answer set is usually generated based on business data in the e-commerce transaction platform, and can also be verified based on internal and external knowledge bases (such as a priori knowledge base or a third-party trusted platform). Specifically, step 204 includes: 204-1. Obtain an initial answer set, where the initial answer set includes candidate answers that match each search field in the target search field set.

[0095] 204-2. Call a priori knowledge base or a third-party trusted platform to calculate the confidence of each answer in the initial answer set.

[0096] 204-3. Remove candidate answers with confidences lower than a preset confidence from the initial answer set to obtain the target answer set.

[0097] For reference Figure 2g A scenario diagram is shown, where the customer service system also includes a response system, and the response system includes a prior knowledge base. It can be seen that before outputting the initial answer set, the confidence of each answer in the initial answer set is calculated by calling the prior knowledge base or a third-party trusted platform to verify the credibility of each conversation, rather than directly outputting it as an answer. Instead, false information and wrong information that are not answered are filtered out, thereby avoiding outputting false information fabricated by the intelligent customer service large model.

[0098] 205. According to a preset strategy, the target answer set is sent and displayed on the first conversation interface.

[0099] It can be seen that in the embodiment of the present application, since the intelligent customer service big model is based on multimodality, it can understand any modal data input by the user. Therefore, on the basis of preliminarily identifying the user's intention based on the first key feature in the first input, the generative model is used to enhance the content based on the user's intention, and a plurality of search fields are obtained. Since these search fields are all semantically associated with the first key feature, on the one hand, the plurality of search fields can further determine the user's intention, so that the target answer set obtained based on the plurality of search fields can better match the user's intention. On the other hand, when the intelligent customer service big model and the user have a real-time conversation, the user does not need to change the content of the input query multiple times to express his or her demands, which significantly reduces the user's inquiry + answer acquisition time, and significantly improves the quality inspection efficiency and the reception quality of the intelligent customer service big model, saving labor costs.

[0100] Optionally, in some embodiments of the present application, for the intelligent customer service big model deployed on the e-commerce transaction platform, since the training set of the intelligent customer service big model can be trained based on part or all of the business data and user data, the business type that the intelligent customer service big model can respond to can also be a dedicated or public intelligent customer service big model covering all business types.

[0101] The following will introduce how the intelligent customer service model in these scenario modes can identify user intent based on the natural language input by the target user and output effective answers from the dimensions of verifying the correctness of the answer, the authenticity of the order, recommending or querying the resources to be purchased, querying the after-sales data, querying the cross-platform distribution, and querying the financial data of resource transactions.

[0102] Scenario 1: Order authenticity verification. Before the intelligent customer service model outputs an order query answer, it can verify the authenticity first. Specifically, the user's input is to query historical orders in the first historical period, such as Figure 3a As shown, the method also includes: 301. Obtain a first input of a target user in a first conversation interface.

[0103] The first input is to query historical orders within a first historical period.

[0104] 302. Obtain a first key feature associated with a resource in the e-commerce transaction platform from the first input.

[0105] 303. Perform content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes a plurality of search fields that are semantically associated with the first key feature.

[0106] 304. Obtain a target answer set, wherein the target answer set includes a first candidate answer to the order list of the target user in the first historical period.

[0107] 305. Traverse the e-commerce transaction system; 306. If it is verified that the order information in the candidate answer does not belong to a real transaction order in the e-commerce transaction system, the candidate answer that does not belong to the real transaction order is removed from the initial answer set to obtain the target answer set; 307. Display a first answer including the target answer set on the first conversation interface.

[0108] For example, if a user asks "I want to check shopping orders within 1 week", the intelligent customer service 001 that does not adopt the technical solution of this application may give the following answer: Figure 3b As shown in Fig. 1, the order numbers are machine-generated numbers with a very regular pattern. For an e-commerce trading platform, the order numbers generated when the same user or different users place orders at the same time or at different times are usually not consecutive and similar. Figure 3b According to the rule in fig1, as long as the user clicks or copies it to query, the result that the order number is empty can be basically obtained. When the intelligent customer service using the technical solution of this application faces the same inquiry, the answer given is reference Figure 3b From the schematic diagram of fig2 in the figure, we can see that fig2 conforms to the normal rules, and when the user clicks the corresponding position of any order number in the conversation interface, he can directly enter the order interface corresponding to the order. It can be seen that the answer of the intelligent customer service 001 in fig2 can be verified and is credible.

[0109] It can be seen that by traversing the e-commerce transaction system to verify the actual transaction orders of the target user before outputting the order query answer, the invalid orders output by the intelligent customer service model due to attacks or the need to improve robustness can be reduced or avoided. Usually, the invalid orders are fabricated by the intelligent customer service model itself, and the fabricated invalid orders may look real and cannot be distinguished without careful observation. This will confuse the real order data and bring wrong reference data to the real-time query of the target user. Therefore, after verifying the authenticity of the order, the candidate answers that include non-real transaction orders in the initial answer set are removed. In this way, the authenticity of the candidate answers presented to the target user for browsing in the first conversation interface is guaranteed, and the query efficiency and user experience are also improved.

[0110] Scenario 2: Recommendation or query of resources to be purchased Before the intelligent customer service model outputs the query answer for the product to be purchased, it can also understand the user's desired product, budget, preferences and other dimensional information. Specifically, the user's input is to query or recommend the target user's to-be-purchased resources in the second historical period, such as Figure 4 As shown, the method also includes: 401. Obtain a first input of a target user in a first conversation interface.

[0111] The first input is to query or recommend resources to be purchased by the target user in the second historical period.

[0112] 402. Obtain a first key feature associated with a resource in the e-commerce transaction platform from the first input.

[0113] 403. Perform content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes a plurality of search fields that are semantically associated with the first key feature.

[0114] 404. Obtain a target answer set, wherein the target answer set includes a second candidate answer to the target user's list of resources to be purchased in the second historical period.

[0115] 405. Determine resource type and purchase intention according to the first key feature.

[0116] 406. Obtain candidate resources to be purchased that match the resource type from the shopping cart of the target user.

[0117] 407. Determine a list of resources to be purchased from the candidate resources to be purchased according to the shopping habits and purchase intentions of the target user.

[0118] 408. Display a second candidate answer including the list of resources to be purchased in the first conversation interface.

[0119] For example, a user sends a message in the conversation interface saying "I want to buy a business laptop computer that is suitable for gaming and office work and has a long battery life". The intelligent customer service model retrieves the user's shopping cart and traverses the entire platform based on the user's query. After comparing the parameters, it recommends multiple notebook recommendation links to the user for selection. It can be seen that by screening out a list of resources to be purchased that meet the shopping habits and purchasing intentions of the target user from the candidate resources to be purchased based on the shopping habits and purchasing intentions of the target user, and then recommending them to the target user, it can better match the user's purchasing intention, thereby increasing the probability of the target user purchasing the product, and also reducing the time spent by the user browsing and searching one by one.

[0120] Scenario 3: Query of after-sales data In this scenario mode 3, the target user only needs to input data in any mode such as natural language, pictures, voice, video, etc. in the first conversation interface, and the intelligent customer service model can intelligently output the query answer for after-sales data. Specifically, the user's input is to query the after-sales data of the target user in the third historical period, such as Figure 5a As shown, the method also includes: 501. Obtain a first input of a target user in a first conversation interface.

[0121] The first input is to query the after-sales data of the target user in the third historical period.

[0122] 502. Obtain a first key feature associated with a resource in the e-commerce transaction platform from the first input.

[0123] 503. Perform content enhancement on the first key feature based on the generative model to obtain a target search field set.

[0124] The target search field set includes a plurality of search fields that are semantically associated with the first key feature.

[0125] 504. Obtain a target answer set.

[0126] The target answer set includes third candidate answers for the after-sales list of the target user in the third historical period.

[0127] 505. Determine the after-sales resource type and after-sales type according to the first key feature; 506. Obtain candidate after-sales data matching the after-sales resource type and after-sales type from the shopping cart of the target user.

[0128] 507. Determine an after-sales list from the candidate after-sales data according to the after-sales quantity and after-sales status of the resources.

[0129] 508. Display a third candidate answer including the after-sales list in the first conversation interface.

[0130] Among them, the third candidate answer is a first visualization chart, which includes a dynamic switching icon; the first visualization chart is obtained according to after-sales orders, after-sales time, after-sales category, after-sales status, after-sales results and after-sales personnel.

[0131] For example, if Figure 5b As shown in the figure, the user (including distribution suppliers, store sellers, e-commerce trading platform after-sales, etc.) enters "after-sales volume and after-sales status of X2 model CPU in the last month" in the conversation interface. The intelligent customer service model retrieves the order-related information of X2 model CPU sold by the user within 1 month, generates the first visual chart after status analysis and statistics, and sends it to the conversation interface for the user to click and view at any time.

[0132] It can be seen that users do not need to select filtering conditions to export after-sales data, and then download it to the local area for analysis and report making. With this solution, the intelligent customer service model can understand the user's statistical needs for after-sales data based on a natural language input by the user, and analyze and count the actual order data, and make it into a visual chart output. Obviously, it effectively improves the efficiency of users in analyzing after-sales situations, is convenient, and has a high user experience.

[0133] Scenario 4: Cross-platform distribution query Before outputting the query answer of the product to be distributed, the transaction status is confirmed first. Specifically, the user input is to query the distribution resource transaction data of the target user on different e-commerce trading platforms in the fourth historical period, such as Figure 6a As shown, the method also includes: 601. Obtain a first input of a target user in a first conversation interface.

[0134] Among them, the first input is to query the distribution resource transaction data of the target user on different e-commerce trading platforms during the fourth historical period.

[0135] 602. Obtain a first key feature associated with a resource in the e-commerce transaction platform from the first input.

[0136] 603. Perform content enhancement on the first key feature based on the generative model to obtain a target search field set. The target search field set includes multiple search fields that are semantically associated with the first key feature.

[0137] 604. Obtain a target answer set.

[0138] The target answer set includes a fourth candidate answer to the target user's list of distributed resources within the fourth historical period.

[0139] 605. Obtaining first transaction data of the target user's distributed resources on different e-commerce trading platforms; 606. Determine first description information of the resource to be distributed according to the first transaction data, where the first description information includes information of the resource to be distributed and an e-commerce transaction platform to be distributed; 607. Determine a resource distribution strategy according to the first description information; 608. Displaying a fourth candidate answer including the resource distribution strategy in the first conversation interface; Among them, the fourth candidate answer is a second visualization chart, and the second visualization chart includes a smart distribution application icon and information on resources to be distributed on different e-commerce trading platforms.

[0140] For example, if Figure 6b As shown in the figure, the distribution supplier enters "Help me choose the right way to distribute products on different e-commerce trading platforms" in the conversation interface. The intelligent customer service model retrieves the order-related information of all products of the user within 3 months, including transactions, after-sales, user portraits of different e-commerce trading platforms, and positioning of the products to be distributed (including information management, inventory, sales portraits, etc. of all products), and generates a second visual chart after analysis and statistics, which is sent to the conversation interface for the user to click and view at any time.

[0141] It can be seen that users do not need to first pull the transaction data of goods that have been distributed on different e-commerce trading platforms, nor do they need to manually select screening conditions to export, and then download to the local area for analysis and report making. With this solution, the intelligent customer service model can understand the user's statistical needs for distribution data based on a natural language input by the user, and analyze and count the actual order data and goods to be distributed, and make them into visual charts for output. Obviously, it effectively improves the order processing process, selects the appropriate e-commerce trading platform for classified distribution, and solves problems such as inventory synchronization. The overall distribution efficiency is high, and informatization and intelligence improve the rationality of distribution, convenience, and high user experience.

[0142] Scenario 5: Financial data query of resource transactions Output the query answer of financial statements based on a natural language input by the user. Specifically, the user input is to query the financial data of the target user on different e-commerce trading platforms in the fifth historical period, such as Figure 7 As shown, the method also includes: 701. Obtain a first input of a target user in a first conversation interface, wherein the first input is to query the financial data of the target user on different e-commerce trading platforms in a fifth historical period.

[0143] 702. Obtain a first key feature associated with a resource in the e-commerce transaction platform from the first input.

[0144] 703. Perform content enhancement on the first key feature based on the generative model to obtain a target search field set.

[0145] The target search field set includes a plurality of search fields that are semantically associated with the first key feature.

[0146] 704. Obtain a target answer set, wherein the target answer set includes a fifth candidate answer of the financial list of the historical transaction resources of the target user in the fifth historical period.

[0147] 705. Obtain second transaction data of resource transactions of the target user on different e-commerce transaction platforms.

[0148] 706. Determine financial data of historical transaction resources according to the second transaction data.

[0149] 707. Generate financial statements based on the financial data.

[0150] 708. Display a fifth candidate answer including the financial statement in the first conversation interface.

[0151] Among them, the fifth candidate answer is a third visualization chart, and the third visualization chart includes a dynamic switching icon; the third visualization chart is obtained based on historical transaction orders, settlement data and fund flow.

[0152] For example, if Figure 8 As shown in the figure, the user (including distribution suppliers, store sellers, after-sales service of e-commerce trading platforms, etc.) enters "transaction financial data of the store in the last three months" in the conversation interface. The intelligent customer service model retrieves the order-related information of all products that the user has traded and after-sales services in within three months, and generates a second visual chart after status analysis and statistics, which is sent to the conversation interface for the user to click and view at any time.

[0153] It can be seen that users do not need to select filtering conditions to export their own financial data on the e-commerce trading platform, and then download it to the local area for analysis and report making. With this solution, the intelligent customer service model can understand the user's statistical needs for financial data based on a natural language input by the user, and analyze and count the actual order data, and make it into a visual chart output. Obviously, it effectively improves the efficiency of users analyzing income and expenditure, is convenient, and has a high user experience.

[0154] It is understandable that the above five scenario modes are only introduced based on the existing business data, and can be extended to any other scenario modes. The above scenario modes can appear successively in the same session or in different sessions, and the embodiments of the present application do not limit this. In some ways, in the order search system, risk control can be added during important operations such as distribution and order operation (for example, the degree of danger of the operation can be identified through the model), and then the user can confirm it again to avoid deviations in important operations.

[0155] The conversation processing device in the embodiment of the present application has the function of implementing the intelligent customer service conversation processing method and the training method of the intelligent customer service large model provided in the above embodiment. The functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0156] See also Fig. 9 , Fig. 9 The structure diagram of the conversation processing device 50 provided in the embodiment of the present application can be applied to a computing device in a scenario where a conversation with a user needs to be implemented based on a large model, for example, it can be applied to a customer service system that processes the conversation between the user and the intelligent customer service large model. Specifically, the conversation processing device 50 can deploy at least one adversarial generation model 501 (optional), at least one intelligent customer service large model 502, an input and output module 503, a display module 504, and a processing module 505. Among them, the processing module 505 can be used to Figure 1a-Figure 8 In any of the embodiments, operations such as training the intelligent customer service big model 502, determining the target questions and emotional data of the target user, and controlling the operations of the adversarial generation model 501, the intelligent customer service big model 502, the input and output module 503, and the display module 504 are carried out. The same is true for other big models. The adversarial generation model 501, the intelligent customer service big model 502, the display module 504, and the processing module 505 can be deployed separately in a computing device or centrally in a computing device, which is not limited in this application. In some implementations, the display module 504 is used to display the first conversation interface between the intelligent customer service big model and the target user; An input-output module 503 is used to obtain a first input of a target user in a first conversation interface; The processing module 505 is used to obtain a first input from a target user in a first conversation interface; obtain a first key feature associated with a resource in an e-commerce transaction platform from the first input; perform content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes multiple search fields that are semantically associated with the first key feature; obtain a target answer set, wherein the target answer set includes candidate answers that match each search field in the target search field set; and according to a preset strategy, send the target answer set and display it in the first conversation interface through the display module 504.

[0157] Other functions of the adversarial generation model 501, the intelligent customer service model 502, the input and output module 503, the display module 504 and the processing module 505 can be referred to in Figure 1a-Figure 8 The corresponding embodiments are not described in detail.

[0158] It can be seen that this application can further determine the user's intention, so that the target answer set obtained based on the multiple search fields can better match the user's intention. Users do not need to change the content of the input query multiple times to express their demands, which significantly reduces the time for users to ask questions and get answers, and significantly improves the quality inspection efficiency and the reception quality of the intelligent customer service model.

[0159] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems, devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0160] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0161] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0162] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0163] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

Claims

1. An intelligent customer service conversation processing method, characterized in that: The method is applied to an e-commerce customer service system based on a multimodal large model, and the method comprises: Obtaining a first input of a target user in a first conversation interface; Acquire a first key feature associated with a resource in the e-commerce transaction platform from the first input; Performing content enhancement on the first key feature based on a generative model to obtain a target search field set, wherein the target search field set includes a plurality of search fields that are semantically associated with the first key feature; Acquire a target answer set, the target answer set including candidate answers matching each search field in the target search field set; According to a preset strategy, the target answer set is sent and displayed on the first conversation interface.

2. The method according to claim 1, characterized in that The acquisition target answer set includes: Acquire an initial answer set, the initial answer set including candidate answers matching each search field in the target search field set; Calling a priori knowledge base or a third-party trusted platform to calculate the confidence of each answer in the initial answer set; Candidate answers with confidence levels lower than a preset confidence level are removed from the initial answer set to obtain the target answer set.

3. The method according to claim 2, characterized in that The first input is a query for historical orders within a first historical period, and the target answer set includes a first candidate answer to the target user's order list within the first historical period; The method further comprises: Traverse the e-commerce transaction system; If it is verified that the order information in the candidate answer does not belong to the real transaction order in the e-commerce transaction system, the candidate answer that does not belong to the real transaction order is removed from the initial answer set to obtain the target answer set; A first answer including the target answer set is displayed on the first conversation interface.

4. The method according to claim 3, characterized in that The first input is to query or recommend resources to be purchased by the target user in a second historical period, and the target answer set includes a second candidate answer to the list of resources to be purchased by the target user in the second historical period; the method further includes: Determining resource type and purchase intention based on the first key feature; Acquire candidate resources to be purchased that match the resource type from the shopping cart of the target user; Determining a list of resources to be purchased from the candidate resources to be purchased according to the shopping habits and purchase intentions of the target user; A second candidate answer including the list of resources to be purchased is displayed in the first conversation interface.

5. The method according to any one of claims 1 to 4, characterized in that The first input is to query the after-sales data of the target user in a third historical period, and the target answer set includes a third candidate answer of the after-sales list of the target user in the third historical period; the method further includes: Determine the after-sales resource type and after-sales type according to the first key feature; Acquire candidate after-sales data matching the after-sales resource type and after-sales type from the shopping cart of the target user; Determine an after-sales list from the candidate after-sales data according to the after-sales quantity and after-sales status of the resources; Displaying a third candidate answer including the after-sales checklist in the first conversation interface; Among them, the third candidate answer is a first visualization chart, which includes a dynamic switching icon; the first visualization chart is obtained according to after-sales orders, after-sales time, after-sales category, after-sales status, after-sales results and after-sales personnel.

6. The method according to claim 5, characterized in that The first input is to query the distribution resource transaction data of the target user on different e-commerce trading platforms in a fourth historical period, and the target answer set includes a fourth candidate answer of the list of distributed resources of the target user in the fourth historical period; the method further includes: Obtain the first transaction data of target users on the resources that have been distributed on different e-commerce trading platforms; Determine first description information of the resource to be distributed according to the first transaction data, wherein the first description information includes information of the resource to be distributed and an e-commerce transaction platform to be distributed; Determine a resource distribution strategy according to the first description information; Displaying a fourth candidate answer including the resource distribution strategy in the first conversation interface; Among them, the fourth candidate answer is a second visualization chart, and the second visualization chart includes a smart distribution application icon and information on resources to be distributed on different e-commerce trading platforms.

7. The method according to claim 6, characterized in that The first input is to query the target user's financial data on different e-commerce trading platforms in a fifth historical period, and the target answer set includes a fifth candidate answer of a financial list of historical transaction resources of the target user in the fifth historical period; the method further includes: Acquire second transaction data of resource transactions of target users on different e-commerce transaction platforms; determining financial data of historical transaction resources according to the second transaction data; generating financial statements based on the financial data; displaying a fifth candidate answer including the financial statement in the first conversation interface; Among them, the fifth candidate answer is a third visualization chart, and the third visualization chart includes a dynamic switching icon; the third visualization chart is obtained based on historical transaction orders, settlement data and fund flow.

8. The method according to claim 7, characterized in that The e-commerce customer service system further includes an adversarial generation model and a query generation model, and the method further includes: Obtaining a first prompt word input into the adversarial generation model; generating a first set of questions for an e-commerce trading platform based on the first prompt word, wherein the first set of questions includes a plurality of adversarial e-commerce questions; The first question set is input into the query generation model to perform adversarial training on the query generation model.

9. A conversation processing device, characterized in that: The session processing device comprises: at least one processor and memory; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method according to any one of claims 1 to 8.

10. A computer program product comprising instructions, the computer program product comprising program instructions, which, when executed on a computer or a processor, enable the computer or the processor to perform the method according to any one of claims 1 to 8.

Images