Customer service interaction method, interaction device, equipment, storage medium and program product

Abstract

The invention provides a customer service interaction method, an interaction device, equipment, a storage medium and a program product, and relates to the technical field of computers. The customer service interaction method based on the large language model comprises the following steps: performing intention scene recognition on a received user question; identifying that the user question belongs to a specified intention scene, and determining a corresponding domain knowledge base matched with the user question; performing enhanced retrieval on the knowledge base in the corresponding field based on the user question to obtain a business knowledge fragment for enhancing the user question; and inputting the business knowledge fragments into a large language model, so that the large language model performs filtering type question reply processing based on the business knowledge fragments, and outputs reply information of the user question based on a processing result. According to the technical scheme, the real intention of the user in continuous interaction can be deeply analyzed, the content which the user wants to consult can be more accurately grasped, and the consulting requirement of the user is better met.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a customer service interaction method based on a large language model, a customer service interaction device based on a large language model, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] The customer service intelligent question and answer system is a system used to automatically answer questions raised by customers in customer service scenarios. The system processes user query requests based on predefined rules, keyword matching, and database queries, and provides responses. However, due to the inability to effectively understand more complex natural language expressions, it often fails to correctly understand the user's intentions, resulting in inflexible responses to complex requests and a poor experience for problem users.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a customer service interaction method based on a large language model, a customer service interaction device based on a large language model, an electronic device, a computer-readable storage medium and a computer program product, which at least to a certain extent overcome the problem in the related art that the customer service intelligent question and answer system is not flexible enough in responding to complex requests, resulting in a poor experience for the user asking the question.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a customer service interaction method based on a large language model is provided, including: performing intent scenario recognition on received user questions; identifying that the user questions belong to specified intent scenarios, and determining a corresponding domain knowledge base that matches the user questions; performing enhanced retrieval on the corresponding domain knowledge base based on the user questions to obtain business knowledge fragments that enhance the user questions; inputting the business knowledge fragments into the large language model so that the large language model performs filtered question answering processing based on the business knowledge fragments, and outputting answer information for the user questions based on the processing results.

[0007] In one embodiment of the present disclosure, the business knowledge fragment includes a business background title and business background content, the large language model includes a front filtering model, a back filtering model and a main language model, the business knowledge fragment is input into the large language model, so that the large language model performs filtering question answering processing based on the business knowledge fragment, and outputs the answer information of the user question based on the processing result, including: inputting the business background title and the user question into the front filtering model, so as to detect the subject consistency between the business background title and the user question based on the front filtering model; if the subject is consistent, inputting the user question and the business background content into the main language model to generate information to be answered; inputting the information to be answered and the user question into the back filtering model, so as to detect the correlation between the information to be answered and the user question based on the back filtering model; if it is determined to be related, the information to be answered is determined as the answer information and fed back to the user end.

[0008] In one embodiment of the present disclosure, before performing intent scenario recognition on a received user question, it includes: detecting whether the user's session history question-answer pair is empty based on the user identifier of the user question, so as to determine whether the received user question belongs to a multi-round question or a first-round question, wherein if the session history question-answer pair is empty, it is determined that the user question belongs to the first-round question, and if the session history question-answer pair is not empty, it is determined that the user question belongs to the multi-round question.

[0009] In one embodiment of the present disclosure, identifying that the user question belongs to a specified intent scenario and determining a corresponding domain knowledge base that matches the user question include: determining that the received user question belongs to a multi-round question, the user question belongs to the specified intent scenario, and determining the specified historical questions in the multi-round questions that belong to the specified intent scenario; querying the knowledge base consulted when replying to the specified historical questions as the corresponding domain knowledge base, wherein the historical questions in the multi-round questions that have the shortest time interval with the user question and belong to the specified intent scenario are determined as the specified historical questions.

[0010] In one embodiment of the present disclosure, an enhanced search is performed on the corresponding domain knowledge base based on the user question to obtain a business knowledge fragment enhanced for the user question, including: generating information to be searched based on the name of the corresponding domain knowledge base, the user question and the specified historical question; and searching the corresponding domain knowledge base based on the information to be searched to obtain the business knowledge fragment enhanced for the user question.

[0011] In one embodiment of the present disclosure, information to be retrieved is generated based on the name of the corresponding domain knowledge base, the user question and the specified historical question, including: obtaining an intermediate historical question input between the user question and the specified historical question; inputting the user question, the intermediate historical question and the name of the corresponding domain knowledge base into a query rewriting model for rewriting to obtain the information to be retrieved.

[0012] In one embodiment of the present disclosure, identifying that the user question belongs to a specified intent scenario, and determining a corresponding domain knowledge base that matches the user question, also includes: determining that the user question is a first-round question, and identifying that the first-round question belongs to the specified intent scenario, and traversing the alternative knowledge bases to select the corresponding domain knowledge base that matches the first-round question domain.

[0013] In one embodiment of the present disclosure, an enhanced search is performed on the corresponding domain knowledge base based on the user question to obtain a business knowledge fragment enhanced for the user question, including: searching the corresponding domain knowledge base based on the first-round question to obtain context information enhanced for the user question; and concatenating the context information and the first-round question as the enhanced business knowledge fragment.

[0014] In one embodiment of the present disclosure, the corresponding domain knowledge base is retrieved based on the first-round question to obtain context information enhanced for the user question, including: rewriting the first-round question input into a query rewriting model to obtain a rewritten question; inputting the rewritten question into a question-answer pair library for matching; if there is a match, generating the reply information; if there is no match, retrieving the corresponding domain knowledge base based on the rewritten question to obtain the context information enhanced for the user question.

[0015] In one embodiment of the present disclosure, the received user questions are subjected to intent scenario identification, further comprising: if it is identified that the first round of questions do not belong to the specified intent scenario, locating the question intent to which the first round of questions belong; determining a fixed speech library configured for the question intent based on the locating results; and querying the answer information matching the first round of questions from the fixed speech library.

[0016] In one embodiment of the present disclosure, it also includes: determining that the received user question belongs to a multi-round question, and the user question does not belong to the specified intention scenario, then processing the user question as the first-round question that does not belong to the specified intention scenario.

[0017] In one embodiment of the present disclosure, before detecting whether the user's session history question and answer pair is empty based on the user identifier of the user question, it also includes: in response to the received user question, performing answer semantic matching on the user question based on the robot knowledge base, wherein if there is a matching result, the reply information is generated, and if there is no matching result, detecting whether the user's session history question and answer pair is empty.

[0018] In one embodiment of the present disclosure, it also includes: the query rewriting model is rewritten based on at least one of similar word expansion, knowledge graph association and similar semantic retrieval data sub-models.

[0019] In one embodiment of the present disclosure, before performing intent scenario recognition on a received user question, it also includes: in response to the received original question information of the user, if it is detected that the original question information includes a data table, normalizing the data table to obtain a normalized table, wherein the normalized table has a first number of rows and a second number of columns; disassembling and semantically converting the normalized table based on the dimensions of the rows and the dimensions of the columns to obtain the user question; if the original question information does not include the data table, determining the original question information as the user question.

[0020] In one embodiment of the present disclosure, intent scenario recognition is performed on received user questions, and the method further includes: performing intent scenario recognition on the user questions based on an intent scenario recognition model, and outputting probability distributions of different intent scenario categories; if the designated intent scenario has the highest probability in the probability distribution, it is determined that the designated intent scenario belongs to the designated intent scenario, wherein the designated scenario intent includes a consultation category, and the non-designated scenario intent includes at least one of an inquiry category, a handling category, a fault reporting category, and a complaint category.

[0021] In one embodiment of the present disclosure, before performing intent scenario recognition on received user questions, it also includes: performing data cleansing on the acquired historical question and answer data of the corresponding field to obtain processed data; performing block processing on the processed data based on a key-value pair relationship to obtain data blocks, the data blocks including a business background title and business background content, the business background title corresponds to a key, and the business background content corresponds to a value; constructing a knowledge base index based on the key to generate the corresponding field knowledge base based on the knowledge base index and the data blocks.

[0022] According to another aspect of the present disclosure, a customer service interaction device based on a large language model is provided, comprising: an identification module, used to perform intent scenario identification on a received user question; a determination module, used to identify that the user question belongs to a specified intent scenario, and determine a corresponding domain knowledge base that matches the user question; a retrieval module, used to perform an enhanced retrieval of the corresponding domain knowledge base based on the user question to obtain a business knowledge fragment enhanced for the user question; and a reply module, used to input the business knowledge fragment into the large language model so that the large language model performs filtered question reply processing based on the business knowledge fragment, and outputs reply information for the user question based on the processing result.

[0023] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; the processor is configured to execute the above-mentioned customer service interaction method based on a large language model by executing the executable instructions.

[0024] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the customer service interaction method based on the large language model is implemented.

[0025] According to another aspect of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the customer service interaction method based on the large language model is implemented.

[0026] The customer service interaction solution based on the large language model provided by the embodiment of the present disclosure can deeply analyze the real intention of the user in the continuous interaction by determining multiple rounds of questions and the designated intention scenarios to which they belong, which is conducive to more accurately grasping the content that the user wants to consult, determining the name of the corresponding domain knowledge base according to the designated historical questions, and generating information to be retrieved to retrieve the knowledge base to obtain business knowledge fragments. The business knowledge fragments can include user intention information and related context information. The business knowledge fragments are input as the large language model so that the large language model outputs reply information, which is not only accurate in business content, but also more natural and fluent in language expression, improving the overall reply quality, better meeting user consultation needs, and enhancing user satisfaction with the customer service intelligent question and answer system. In addition, by configuring the corresponding domain knowledge base, a large language model based on retrieval enhancement is generated, and a customer service intelligent interaction system is constructed based on the large language model based on the retrieval enhancement. The large language model based on retrieval enhancement has the advantages of low cost and efficient reply. The large model technology is applied to the field of telecommunications customer service, which greatly improves the intelligence and interactivity of the system and reduces the investment in customer service manpower.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0029] Figure 1 A schematic diagram showing a customer service interaction system based on a large language model in an embodiment of the present disclosure is shown;

[0030] Figure 2 A schematic flow chart showing a customer service interaction method based on a large language model in an embodiment of the present disclosure is shown;

[0031] Figure 3 A schematic flow chart showing another customer service interaction method based on a large language model in an embodiment of the present disclosure;

[0032] Figure 4 A schematic flow chart showing another customer service interaction method based on a large language model in an embodiment of the present disclosure is shown;

[0033] Figure 5 A schematic flow chart showing another customer service interaction method based on a large language model in an embodiment of the present disclosure;

[0034] Figure 6 A schematic flow chart showing another customer service interaction method based on a large language model in an embodiment of the present disclosure;

[0035] Figure 7 A schematic flow chart showing another customer service interaction method based on a large language model in an embodiment of the present disclosure;

[0036] Figure 8 A schematic diagram of a customer service interaction device based on a large language model in an embodiment of the present disclosure is shown;

[0037] Fig. 9 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0038] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0039] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0040] The customer service intelligent question-and-answer system can be implemented based on predetermined rules. However, this method cannot effectively understand complex natural language expressions because it relies on keyword matching and fixed rules. It often cannot correctly understand user intentions for questions with diverse expressions and implicit information. In addition, it cannot handle multiple rounds of conversations because it lacks context tracking capabilities. When users interact multiple times in a conversation, the system cannot remember previous inputs, causing users to repeatedly input information, affecting the experience.

[0041] In addition, applying the big language model to the customer service intelligent question-answering system can overcome many limitations of the above-mentioned customer service intelligent question-answering system based on its powerful natural language processing and generation capabilities. It can understand complex languages, support multiple rounds of dialogue, generate personalized answers, handle unpredicted questions, and provide a more natural and flexible user experience. By integrating the big model, the customer service system can be more intelligent and efficient, thereby significantly improving the quality of customer service.

[0042] However, since the customer service intelligent question-and-answer system has the professionalism of its field, taking the telecommunications customer service intelligent question-and-answer system as an example, it usually needs to be developed around telecommunications products and services. The open source general large model lacks professional data in the telecommunications customer service field when training the model, resulting in the direct use of the open source model cannot make it play its advantages in the telecommunications customer service field. In addition, fine-tuning or fine-tuning the large model requires a large amount of data annotation and a lot of GPU resources, which requires a large investment of manpower and financial resources.

[0043] In order to apply the language big model to the specified customer service field, such as the telecommunications customer service field, and reduce the cost of fine-tuning and training, a low-cost and efficient solution is to use the knowledge base of the corresponding field, such as the knowledge base in the telecommunications field, combined with the existing big model capabilities, reduce the dependence on data annotation and GPU resources, and fully combine RAG (Retrieval-Augmented Generation) with the big model, which can provide an intelligent and low-cost solution in the telecommunications customer service system.

[0044] To facilitate understanding, several terms involved in this application are first explained below.

[0045] RAG (Retrieval-Augmented Generation): is a model architecture that combines information retrieval and text generation. It enhances the output of the generative model by retrieving relevant information (such as documents or data fragments), thereby improving the accuracy and relevance of answers. RAG is often used in question-answering systems and dialogue generation, leveraging information from external knowledge bases to support the generation of richer and more accurate content.

[0046] LLM (Large Language Model): refers to a language model trained using deep learning technology, which is usually very large in size, containing hundreds of millions to trillions of parameters. These models are trained with large amounts of text data and can understand and generate natural language. LLM performs well in a variety of natural language processing tasks, including text generation, translation, question answering, dialogue systems, etc.

[0047] Figure 1 1 is a schematic diagram of a customer service interaction system provided by an exemplary embodiment of the present application. The system includes: a plurality of user terminals 120 and a server terminal 140, wherein the user terminal 120 sends a user question and the server terminal 140 responds.

[0048] The user end 120 can be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the user end 120 can also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0049] The user terminal 120 may be installed with an application for providing customer service interaction based on a large language model.

[0050] The user end 120 is connected to the server end 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0051] The server 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server 140 is used to provide background services for applications that provide customer service interactions based on large language models. Optionally, the server 140 undertakes the main computing work and the user end 120 undertakes the secondary computing work; or, the server 140 undertakes the secondary computing work and the user end 120 undertakes the main computing work; or, the user end 120 and the server 140 adopt a distributed computing architecture for collaborative computing.

[0052] In some optional embodiments, the server 140 is used to store customer service interaction program information based on a large language model.

[0053] Optionally, the logistics user terminals of the application programs installed in different user terminals 120 are the same, or the logistics user terminals of the application programs installed on the two user terminals 120 are logistics user terminals of the same type of application programs on different control system platforms. Based on the different terminal platforms, the specific forms of the logistics user terminals of the application programs may also be different, for example, the logistics user terminals of the application programs may be mobile phone logistics user terminals, PC logistics user terminals, or World Wide Web (Web) logistics user terminals.

[0054] Those skilled in the art will appreciate that the number of the user terminals 120 may be more or less. For example, the terminal may be only one, or the terminal may be dozens or hundreds, or more. The embodiment of the present application does not limit the number and device type of the terminal.

[0055] Optionally, the system may also include a management device ( Figure 1 (not shown), the management device is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0056] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0057] like Figure 2 As shown, according to an embodiment of the present disclosure, a customer service interaction method based on a large language model includes:

[0058] Step S202: perform intent scenario recognition on the received user question.

[0059] In some embodiments, taking the telecommunications customer service intelligent question-and-answer system, i.e., the telecommunications customer service interactive system, as an example, after the telecommunications customer service intelligent question-and-answer system receives the user's question, it performs intent scenario recognition. The telecommunications field involves many business sectors, such as package handling, network fault repair, call fee inquiry, value-added service consultation, etc., all of which belong to different intent scenarios.

[0060] Step S204: Identify that the user question belongs to a specified intent scenario, and determine a corresponding domain knowledge base that matches the user question.

[0061] In some embodiments, after identifying that a user problem belongs to a specified intent scenario, the corresponding domain knowledge base that matches it is searched based on the constructed knowledge base correspondence relationship, such as a network-related knowledge base that includes network troubleshooting methods, solutions to common network problems, different network fault reporting processes, and other related knowledge content, to ensure that subsequent retrieval can be carried out within the appropriate knowledge scope.

[0062] Step S206: Perform an enhanced search on the corresponding domain knowledge base based on the user question to obtain a business knowledge fragment enhanced for the user question.

[0063] In some embodiments, based on the determined corresponding domain knowledge base, enhanced retrieval is performed according to the current user question, and business knowledge fragments that are more targeted and more relevant to answering the user's network question are mined from the knowledge base.

[0064] Step S208: input the business knowledge fragments into the large language model so that the large language model performs filtering question answering processing based on the business knowledge fragments, and outputs answer information of the user's question based on the processing result.

[0065] In some embodiments, filtering question answering processing by a large language model refers to a processing method in which multiple links collaborate to screen user questions based on a large language model, generate responses, and screen again, thereby providing users with more accurate responses.

[0066] In this embodiment, by identifying the intent scenario of the user's question, if the identification result is a specified intent scenario, in the specified intent scenario, based on the determination of the adapted knowledge base, enhanced retrieval and filtered reply processing with the help of a large language model, the reply information of the user's question is obtained, which can not only accurately grasp the direction of the user's consultation, accurately retrieve the business knowledge that can enhance the reply effect, but also quickly generate reply information that meets the user's needs, thereby improving the user's satisfaction and usage experience with the telecommunications customer service intelligent question and answer system.

[0067] In one embodiment of the present disclosure, the business knowledge fragment includes a business background title and business background content, and the large language model includes a front filtering model, a back filtering model and a main language model. The business knowledge fragment is input into the large language model, so that the large language model performs filtering question answering processing based on the business knowledge fragment, and outputs answer information of the user question based on the processing result, including: inputting the business background title and the user question into the front filtering model, so as to detect the subject consistency between the business background title and the user question based on the front filtering model; if the subject is consistent, inputting the user question and the business background content into the main language model to generate information to be answered; inputting the information to be answered and the user question into the back filtering model, so as to detect the correlation between the information to be answered and the user question based on the back filtering model; if it is determined to be related, the information to be answered is determined as the answer information and fed back to the user end.

[0068] like Figure 3 As shown, according to an embodiment of the present disclosure, a customer service interaction method based on a large language model includes:

[0069] Step S302, determine whether the received user question belongs to a multi-round question and the user question belongs to a specified intention scenario, determine the specified historical question in the multi-round question that belongs to the specified intention scenario, and query the name of the corresponding domain knowledge base consulted when answering the specified historical question.

[0070] In some embodiments, the received user question is analyzed to determine whether it is a multi-round question. A multi-round question indicates that the user has interacted with the customer service system for many times, and the current question is generated based on the continuation of the previous communication.

[0071] In addition, it is also necessary to determine whether the user question is in a specified intent scenario. The specified intent scenario can be understood as a scenario where multiple rounds of conversations often occur. For example, in a telecommunications customer service scenario, the specified intent scenario may include a consulting scenario.

[0072] In some embodiments, if it is determined that the user question belongs to a multi-round question and is within a specified intent scenario, by further combing through the historical questions in previous multiple rounds of interactions, the specified historical questions belonging to the specified intent scenario can be found out. For example, in multiple rounds of communication for a business consultation, the historical questions involving the consulting information can be understood as specified historical questions.

[0073] In one embodiment of the present disclosure, determining a designated historical question belonging to a designated intent scenario among multiple rounds of questions includes: determining a historical question among the multiple rounds of questions that has the shortest time interval with the user question and belongs to the designated intent scenario as the designated historical question.

[0074] In this embodiment, the historical scene with the shortest time interval with the current user's question is selected because the scenarios with the same specified intent that have appeared in the recent period are often more highly correlated and can better reflect the current user's conversation ideas and potential needs, so as to extract the past information that is most relevant to the current question, thereby improving the accuracy of the answer and the coherence of the conversation.

[0075] In some embodiments, by determining a specified historical question, the name of the corresponding domain knowledge base that was consulted when answering the specified historical question is searched. The corresponding domain knowledge base refers to a knowledge set constructed for a specific business field. Different business problems will be associated with different knowledge bases. Finding the corresponding knowledge base name is conducive to ensuring that the required knowledge is accurately retrieved subsequently, so as to ensure the reliability of retrieval enhancement.

[0076] Step S304: Generate information to be retrieved based on the name of the corresponding domain knowledge base, the user question and the specified historical question.

[0077] In some embodiments, after obtaining the name of the corresponding domain knowledge base, the information to be retrieved is generated according to certain rules or algorithms in combination with the current user question and the previously determined specified historical question to ensure that the retrieval is more targeted.

[0078] Step S306: searching the corresponding domain knowledge base based on the information to be searched, and obtaining business knowledge fragments that enhance the user's question.

[0079] In some embodiments, the information to be retrieved generated in the previous step is used to perform an enhanced retrieval operation in the corresponding domain knowledge base. Through the retrieval, business knowledge fragments that enhance user questions and have a high degree of match with the information to be retrieved are screened out. The business knowledge fragments enhanced for user questions are partial knowledge related to user questions screened out from the knowledge base, so as to ensure that when the business knowledge fragments enhanced for user questions are used as the input of the language big model for answering questions, the language big model can take into account the contextual information of the question and the specific scope of the question when generating reply information, so as to achieve retrieval enhancement.

[0080] Step S308: input the business knowledge fragment enhanced for the user's question into the large language model, and output the answer information of the user's question.

[0081] In some embodiments, the retrieved business knowledge fragments that enhance the user's questions are input into the large language model. The large language model has powerful language understanding and generation capabilities. It will generate reply information for the user's questions based on the input business knowledge fragments that enhance the user's questions and the language patterns it has learned. For example, after inputting the knowledge fragments about return and exchange conditions retrieved from the after-sales policy knowledge base, the large language model will generate a text content that conforms to the language expression habits and clearly and accurately answers the user's return and exchange questions as reply information.

[0082] In this embodiment, by determining multiple rounds of questions and the designated intent scenarios to which they belong, the user's true intent in continuous interactions can be deeply analyzed, which is conducive to more accurately grasping the content that the user wants to consult, determining the name of the corresponding domain knowledge base based on the designated historical questions, and generating information to be retrieved to search the knowledge base to obtain business knowledge fragments enhanced for user questions. The business knowledge fragments enhanced for user questions can include user intent information and related contextual information. The business knowledge fragments enhanced for user questions are used as input to the large language model so that the large language model outputs reply information, which is not only accurate in business content, but also more natural and fluent in language expression, thereby improving the overall reply quality, better meeting user consultation needs, and enhancing user satisfaction with the customer service intelligent question and answer system.

[0083] In addition, by configuring the corresponding domain knowledge base, a retrieval-enhanced language big model was generated, and a customer service intelligent interaction system was built based on the retrieval-enhanced language big model. The retrieval-enhanced language big model has the advantages of low cost and efficient response. Applying the big model technology to the field of telecommunications customer service has greatly improved the intelligence and interactivity of the system and reduced the investment in customer service manpower.

[0084] In one embodiment of the present disclosure, generating information to be retrieved based on the name of the corresponding domain knowledge base, the user question and the specified historical question includes:

[0085] The intermediate historical questions input between the user question and the specified historical question are obtained; the user question, the intermediate historical question and the name of the corresponding domain knowledge base are input into the query rewriting model for rewriting to obtain the information to be retrieved.

[0086] Among them, the intermediate historical questions input between the user question and the specified historical questions reflect the contextual information of the conversation, which can help the large language model understand what the user has discussed before, so as to maintain coherence when rewriting the current question, so that the rewritten question can better acquire new knowledge related to it based on the existing conversation.

[0087] In this embodiment, by inputting the knowledge base name, related historical questions and current user questions into the query rewriting model, the user questions can be rewritten by integrating various information, so that the rewritten questions are more in line with the knowledge base content and the conversation context, which is beneficial to improving the retrieval effect and the accuracy of the answer information obtained for the user questions.

[0088] like Figure 4 As shown, in one embodiment of the present disclosure, before performing intent scene recognition on the received user question, it also includes:

[0089] Step S402, based on the user identifier of the user question, detect whether the user's session history question and answer pair is empty to determine whether the received user question belongs to a multi-round question or a first-round question. If the session history question and answer pair is empty, it is determined that the user question belongs to a first-round question. If the session history question and answer pair is not empty, it is determined that the user question belongs to a multi-round question.

[0090] In some embodiments, based on the user identifier attached to the user's question, the user's past conversation history question and answer records are searched. The user identifier represents the user identity and is used to accurately distinguish the conversation information of different users. By searching the relevant storage area (such as a database, etc.), it is checked whether there is a question and answer record corresponding to the user identifier. If, after searching, it is found that the corresponding conversation history question and answer pair is empty, it means that this is the first time that the user has asked a question, and the current user question is determined to be a first-round question. If the corresponding conversation history question and answer pair is found, it means that the user has interacted with the system, and the user question is determined to be a multi-round question.

[0091] Step S404: determine that the user's question is a first-round question.

[0092] Step S406: If it is identified that the first round of questions belongs to the specified intention scenario, the candidate knowledge bases are traversed to select the corresponding domain knowledge base that matches the domain of the first round of questions.

[0093] Step S408: searching the corresponding domain knowledge base based on the first round of questions to obtain context information.

[0094] Step S410, concatenate the context information and the first round of questions as enhanced business knowledge fragments, input the enhanced business knowledge fragments into the large language model for answering, and output answer information of the user's question.

[0095] Step S412: if it is identified that the first round of questions do not belong to the specified intention scenario, locate the question intention to which the first round of questions belong.

[0096] In some embodiments, when it is determined that the first round of questions does not belong to the specified intent scenario, it is necessary to further clarify which specific intent category it belongs to, which can be achieved based on natural language processing technology and predefined rules. For example, a keyword-based classification method can be used to sort out a list of common keywords under different intent categories in advance. When certain specific keywords appear in the first round of questions, they are preliminarily divided into the corresponding intent categories; machine learning models can also be used to use a large number of questions with labeled intent categories as training data to train the model so that it learns the mapping relationship from question text features to intent categories, and then the current first round of questions are input into the trained model to output the question intent category to which it belongs.

[0097] Step S414, determining a fixed script library configured for the corresponding question intent based on the positioning result.

[0098] In some embodiments, a special fixed script library is constructed in advance for different question intentions, and the script library stores standard answer scripts for various possible situations under the intention category.

[0099] Step S416, querying the fixed script library for answer information matching the first round of questions.

[0100] In this embodiment, for the first round of questions belonging to the specified intent scenario, the retrieved context information and the first round of questions are spliced ​​in a certain order to form a complete business knowledge fragment that enhances the user's question. The business knowledge fragment that enhances the user's question is passed as input content to the large language model, so that the large language model generates corresponding reply information to answer the user's first round of questions, preventing blind answers or inaccurate responses due to lack of corresponding domain knowledge.

[0101] For the first round of questions that do not belong to the specified intent scenario, the corresponding fixed script library pre-configured for different intent categories is used to query the reply information. There is no need to process complex knowledge retrieval. It is only necessary to locate the question intent according to the established process, find the corresponding fixed script library and match the reply information. This achieves a quick response to the user's first round of questions and reduces user waiting time.

[0102] In one embodiment of the present disclosure, before detecting whether the user's session history question-answer pair is empty based on the user identifier of the user's question, the method further includes:

[0103] In response to the received user questions, the robot performs answer semantic matching on the user questions based on the robot knowledge base. If there is a matching result, a reply message is generated. If there is no matching result, it is checked whether the user's session history question and answer pair is empty.

[0104] In this embodiment, the received user questions are first input into the robot knowledge base, which includes a set of common question and answer pairs in traditional customer service question and answer systems. By adding semantic matching technology, the accuracy of question and answer pair matching can be improved.

[0105] In one embodiment of the present disclosure, the corresponding domain knowledge base is searched based on the first round of questions to obtain context information, including:

[0106] The first round of questions are input into the query rewriting model to rewrite the questions and obtain the rewritten questions; the rewritten questions are input into the question-answer pair library for matching; if they match, the answer information is generated; if they do not match, the corresponding domain knowledge base is retrieved based on the rewritten questions to obtain the context information.

[0107] In some embodiments, for the first round of questions that are determined to belong to the specified intent scenario, their text content is accurately extracted to ensure that the integrity and format of the questions meet the input requirements of the query rewriting model, and they are prepared to be input into the query rewriting model.

[0108] In some embodiments, the query rewriting model may be constructed based on natural language processing technology, which internally uses a variety of rewriting strategies such as vocabulary replacement, syntax adjustment, semantic expansion, etc. to achieve rewriting.

[0109] In some embodiments, by pre-building a question-and-answer pair library, the question-and-answer pair library can store a large number of question-and-answer pairs formed by corresponding standard answers. These question-and-answer pairs cover a variety of common questions and answers related to specified intent scenarios, and are usually organized according to a certain indexing method to enable rapid retrieval and matching. When entering the rewritten question into the question-and-answer pair library, it is necessary to operate according to the retrieval rules and interface specifications set by the question-and-answer pair library.

[0110] In this embodiment, if a question-answer pair matching the rewritten question is found in the question-answer pair library, the corresponding answer part of the question-answer pair is directly extracted and prepared to be returned to the user as reply information; if the rewritten question does not match a suitable question-answer pair in the question-answer pair library, the corresponding domain knowledge base is searched based on the rewritten question to retrieve contextual information related to the question, and a large language model is requested to reply based on the contextual information, so that reply information can be accurately matched at the semantic level to respond to user questions quickly and accurately.

[0111] In one embodiment of the present disclosure, determining that a received user question belongs to a multi-round question further includes:

[0112] If it is determined that the user question does not belong to the specified intent scenario, the user question will be treated as a first-round question that does not belong to the specified intent scenario.

[0113] In some embodiments, a multi-round discriminant semantic model can be built into the customer service interaction system so that the language model in the customer service interaction system has the ability to handle contextual understanding of up to 8 rounds of conversations, and can accurately respond to users' multi-round questions, with an accuracy rate of 90%. Among them, if the user question that appears in the multi-round question and answer does not belong to the specified intent scenario, that is, although the question appears in the multi-round question and answer, it can be answered independently without referring to historical questions.

[0114] In this embodiment, by identifying the intent scenarios of user questions and processing them separately when they belong to the specified intent scenarios and when they do not belong to the specified intent scenarios, the customer service interaction system is significantly improved in terms of the consistency and accuracy of multi-round conversations. Users do not need to repeatedly provide information, and the system can give more natural and coherent answers based on the conversation history. In addition, the enhancement of multi-round question and answer capabilities reduces the need for manual transfer, optimizes the user experience, effectively reduces the burden on manual customer service, and improves the efficiency and intelligence level of the system in complex scenarios.

[0115] In one embodiment of the present disclosure, it also includes: the query rewriting model is rewritten based on at least one of similar word expansion, knowledge graph association and similar semantic retrieval data sub-model.

[0116] In some embodiments, taking the rewriting based on the question-answer library as an example, preset questions can be configured in the question-answer library. When it is detected that the user question is consistent with the preset question, it can flexibly respond to generate semantically similar user questions, significantly improve the matching accuracy of the question-answer library, and improve the user satisfaction of the customer service robot. In the telecommunications customer service interaction system, more than 100 external service skill calls can be supported, including traffic, call charges, package inquiries, etc.

[0117] In one embodiment of the present disclosure, before performing intent scenario recognition on the received user question, the method further includes:

[0118] In response to receiving the original question information of the user, if it is detected that the original question information includes a data table, the data table is normalized to obtain a normalized table, the normalized table has a first number M of rows and a second number N of columns; the normalized table is disassembled and semantically converted based on the row dimension and the column dimension to obtain the user question; if the original question information does not include a data table, the original question information is determined to be a user question.

[0119] In some embodiments, by normalizing the table, vertical, horizontal, and nested table formats are unified into horizontal tables, complex tables that are not m rows × n columns are converted into simple tables of m rows × n columns, and table semantic segmentation technology is used to further split the simple table from two dimensions: table rows and table columns, and the table language of each row and column is converted into natural language.

[0120] In some embodiments, taking the telecommunications customer service interaction system as an example, the documents introducing telecommunications products or services usually contain rich package tariff information, and most of this information is presented in complex tables. The table content structure is diverse, and customers' consulting questions may involve different forms of combined data, including single table content, multi-table comprehensive content, table and text comprehensive content, single-line content, multi-line content, etc. When processing these diverse content requirements, table parsing is achieved through table normalization processing, and accurate answers are generated by intelligently extracting relevant information from multiple tables or texts based on the context and question semantics.

[0121] In this embodiment, after receiving the original question information of the user, normalization processing is performed for the cases where data tables are contained therein to form a normalized table with fixed rows and columns, and the user question is further obtained based on row and column dimension decomposition and semantic conversion. For the case where there is no data table, the user question is directly determined. This processing method not only realizes the unified standardization of data formats, but also improves data utilization efficiency and enhances the universality of problem processing, which ultimately helps to improve the accuracy of answers, better respond to different forms of user questions and generate answers that meet the needs.

[0122] In one embodiment of the present disclosure, determining whether a received user question belongs to a multi-round question and whether the user question belongs to a specified intention scenario further includes:

[0123] Based on the intention scenario recognition model, the intention scenario of the user question is recognized, and the probability distribution of different intention scenario categories is output; if the specified intention scenario has the highest probability in the probability distribution, it is determined to belong to the specified intention scenario, where the specified scenario intention includes the consultation category, and the non-specified scenario intention includes at least one of the inquiry category, processing category, fault reporting category and complaint category.

[0124] In some embodiments, taking the telecommunications customer service interaction system as an example, on the basis of retaining the robot function model, query call interface module, and front-end function module of the telecommunications customer service interaction system, a scenario intention recognition model is added to divide the entire telecommunications customer service scenario into five major categories, including inquiry, processing, consultation, fault reporting, and complaint. Among them, the four categories of inquiry, processing, fault reporting, and complaint can be regarded as non-specified intent scenarios, and user questions can be answered through the fixed script configured in the scenario. Scenario examples: package inquiry and consultation scenarios can be regarded as specified intent scenarios. A large model is generated through the retrieval enhancement of the 1+2LLM mode to answer user questions.

[0125] In this embodiment, the intent scenario recognition model is used to identify the intent scenarios of user problems, and the probability distribution of different intent scenario categories is output. The scenario to which the specified intent scenario belongs is determined based on whether the specified intent scenario in the probability distribution has the highest probability. The intent scenarios are divided into specified intent scenarios including consultation categories and non-specified intent scenarios covering at least one of the inquiry category, processing category, fault reporting category and complaint category. This approach helps to accurately distinguish the different business scenarios to which user problems belong, and then take targeted follow-up processing measures based on the characteristics of the scenarios, thereby improving the accuracy and effectiveness of handling various user problems and optimizing the overall service quality and user experience.

[0126] In some embodiments, the business knowledge segment includes a business background title and business background content, and the large language model includes a pre-filtering model, a post-filtering model and a main language model, that is, a large language model of a 1+2LLM mode.

[0127] like Figure 5 As shown, in one embodiment of the present disclosure, the business knowledge fragments are input into the large language model, so that the large language model performs filtering question answering processing based on the business knowledge fragments, and outputs answer information of the user question based on the processing result, including:

[0128] Step S502: input the business background title and the user question into the pre-filtering model to detect the subject consistency between the business background title and the user question based on the pre-filtering model.

[0129] In some embodiments, the pre-filtering model is used to receive business background titles and user questions, and based on semantic analysis, topic modeling, keyword extraction, etc., to measure whether the business background titles and user questions are consistent at the subject level. By comparing the semantic topics of the two, it is determined whether the topic involved in the business background title can match the topic of the user question, thereby playing a role in preliminary screening and preventing the subsequent use of irrelevant business background content to generate replies.

[0130] Step S504: if the topics are consistent, the user question and business background content are input into the main language model to generate information to be answered.

[0131] In some embodiments, after the pre-filtering model detects that the business background title is consistent with the user question topic, the main language model receives the user question and the corresponding business background content as input information. The main language model conducts in-depth analysis of the input content and integrates relevant knowledge to generate reply information that meets the user's question requirements and fits the business background. It is the key execution link in the entire reply generation process and determines the quality of the final reply information.

[0132] Step S506: If the subject is inconsistent, it is transferred to manual processing.

[0133] Step S508: input the information to be answered and the user's question into the post-filtering model to detect the correlation between the information to be answered and the user's question based on the post-filtering model.

[0134] In some embodiments, the post-filtering model takes effect after the main language model generates the information to be replied to, and is used to take the information to be replied to and the user's question as input, and use semantic understanding, text similarity calculation, etc. to accurately determine the degree of correlation between the information to be replied to and the user's question. For example, by calculating the semantic similarity between the two, if the similarity reaches a certain threshold, it is determined that they are related, thereby ensuring that the reply information finally fed back to the user is closely centered on the question raised by the user, preventing the reply generated by the main language model from deviating from the core of the user's question, and improving the quality of the reply.

[0135] Step S510: If it is determined to be relevant, the information to be replied is determined as reply information and fed back to the user end.

[0136] Step S512: If it is determined to be irrelevant, it is transferred to manual processing.

[0137] In this embodiment, the large language model of the 1+2LLM mode uses the complementary advantages of each model through pre-filtering, main language model generation and post-filtering processes to accurately screen out business background knowledge that matches the user's question for reply generation, and strictly control the relevance of the final reply to the question, so as to prevent the introduction of irrelevant business background information or the generation deviation of the main language model itself from causing content inconsistent with the user's question, thereby making the reply information provided by the entire question-and-answer system to the user more accurate and reliable, greatly improving the reply accuracy of the large model and greatly reducing the hallucination rate.

[0138] In some embodiments, based on the above table normalization processing, the 1+2LLM mode telecommunication customer service intelligent interaction system uses table normalization technology to unify the vertical, horizontal, table nesting and other forms into horizontal tables, and converts complex tables of non-m rows × n columns into simple tables of m rows × n columns. The simple table is further split using table semantic segmentation technology, and is disassembled from the two dimensions of table rows and table columns, and the table language of each row and each column is converted into natural language. During the retrieval process, user questions will be located in a single row or single column of data, assisting the large model to perform accurate question and answer, so that the 1+2LLM mode telecommunication customer service intelligent interaction system can support table contents of various formats and types, and handle user consultations of diverse organizational forms. Whether it is a single table, cross-table integration, or comprehensive content of table and text, the system can intelligently parse and accurately match relevant data to provide users with accurate answers. This capability enables the customer service system to handle customer consultations more efficiently and flexibly when facing complex telecommunication products and service introductions, thereby improving the overall user experience and system response efficiency.

[0139] like Figure 6 As shown, according to another embodiment of the present disclosure, a customer service interaction method based on a large language model includes:

[0140] Step S602: clean the acquired historical question-and-answer data in the corresponding field to obtain processed data.

[0141] Step S604, block processing is performed on the processed data based on the key-value pair relationship to obtain data blocks, where the data blocks include a business background title and business background content, where the business background title corresponds to the key and the business background content corresponds to the value.

[0142] Step S606: construct a knowledge base index based on the key, so as to generate a corresponding domain knowledge base based on the knowledge base index and the data blocks.

[0143] In some embodiments, when constructing a knowledge base, the original data is subjected to data conversion, cleaning, processing and other operations, and then the data is segmented to obtain key-value pairs, where the key is the background title information and the value is the background content information. A knowledge base index is constructed based on the key, and a telecommunications business vector knowledge base and a telecommunications business scalar knowledge base can be obtained.

[0144] In some embodiments, a telecommunications business knowledge base is constructed through data synchronization, conversion, data segmentation, index construction and other technologies, the knowledge base construction process is automated, knowledge is updated regularly, and the potential risk of user complaints is avoided by replying to users with expired product or business information. In addition, with the semantic understanding ability of the big model, the system can accurately retrieve knowledge. In response to the questions entered by the user, the system can complete knowledge location and retrieval from the massive knowledge base with one click, and accurately return relevant knowledge fragments. Based on the reasoning and generation capabilities of the big model, the system can simulate the role of customer service personnel and automatically generate answers with professional customer service words, thereby effectively helping the business to quickly respond to user needs.

[0145] In some embodiments, steps S602 to S606 may be implemented before determining whether the received user question belongs to a multi-round question.

[0146] Step S608: The business background title and user question provided by the corresponding domain knowledge base are input into the pre-filtering model to perform topic consistency detection.

[0147] Step S610: input the business background content and user questions provided by the corresponding domain knowledge base into the main language model to generate information to be answered.

[0148] Step S612, input the information to be answered and the user's question into the post-filtering model to perform a correlation test with the user's question.

[0149] Step S614, if the detection is successful, a reply message is obtained.

[0150] In this embodiment, in the user consultation response process, when the business knowledge fragments enhanced for user questions obtained from the corresponding domain knowledge base are used as the input of the language large model for both the first-round questions and non-first-round questions, the strategy of the 1+2LLM mode is added to reduce the model hallucination rate. First, relevant knowledge fragments are retrieved through user question retrieval, and then the background title information of the user question and the knowledge fragment are input into the pre-filtering LLM. The main function of this model is to judge the subject consistency between the user question and the retrieved multiple background title information through the user question and the retrieved multiple background title information. If they are inconsistent, they are directly transferred to manual operation. If they are consistent, they continue to execute. It is necessary to give the question and the multiple background contents retrieved to the main generation model. After the reply is generated, it is necessary to pass through the post-filtering LLM to further judge the relevance between the user question and the main LLM reply. If it is relevant, the output content of the main LLM is directly replied. If it is not relevant, it is transferred to manual operation.

[0151] like Figure 7 As shown, according to another embodiment of the present disclosure, a customer service interaction method based on a large language model includes:

[0152] Step S 702 , data cleaning and data processing are performed on the obtained original question-answer pairs to obtain data blocks, where the data blocks include corresponding background titles and background contents.

[0153] The background title is used for the pre-filter LLM, and the background content is used for the post-filter LLM.

[0154] Step S 704 , in response to the received user question, a set of common question and answer pairs in the customer service question and answer system is retrieved based on the robot knowledge base.

[0155] If not, check whether the user's conversation history question and answer pair is empty. If it is empty, it is determined to be the first round of dialogue. If it is not empty, it is determined to be a multi-round dialogue.

[0156] Step S 706, in the first round of dialogue, the intent scenario recognition model is used to identify that the user's question does not belong to a consulting scenario, and a fixed script is configured based on the intent scenario and the numerical value of the user's question is called to generate a reply message.

[0157] Step S 708: If it belongs to a consulting scenario, the user question is rewritten to obtain a rewritten question.

[0158] Step S710, matching the question and answer pairs based on the rewritten question, and generating matching answer information.

[0159] Step S 712, if there is no match, perform a knowledge base search, use the search results as context and splice with the instruction as the enhanced business knowledge fragment, input the 1+2LLM mode large model, and return the reply information and the name of the consulted knowledge base.

[0160] Step S 714, if it is not the first round of dialogue, scene recognition is performed based on the intention scene recognition model. If it is not a consultation scene, it is processed as the first round of dialogue.

[0161] Step S 717, if it is a consulting scenario, match it with the preset common non-information question library first.

[0162] Step S718: if matched, a reply message is generated based on the matching result.

[0163] Step S 720 , if no match is found, check the name of the knowledge base consulted in the consulting scenarios involved in the previous rounds, input the name of the knowledge base and the questions in the previous N rounds into the query rewriting model to obtain the rewritten questions.

[0164] Step S 722, search the knowledge base based on the rewritten question, send the retrieved business knowledge fragments that enhance the user's question to the 1+2LLM model, and return the reply information.

[0165] Among them, the intent scenario recognition model is used to identify which type of telecommunications customer service scenario belongs to: inquiry, processing, consultation, fault reporting, and complaint.

[0166] If a question is detected to have tabular information, the table is converted into natural language.

[0167] In this embodiment, an intelligent interactive system for telecommunications customer service is constructed by integrating the big model-based intention recognition technology and the retrieval-enhanced big model technology. User intention recognition is performed based on the big model, and preset questions in the question-answer pair library are efficiently matched from a semantic level based on user input to achieve accurate recognition of user consultation intentions. The intelligent interactive system for telecommunications customer service supports more than 100 external service skill calls, including traffic, call charges, package inquiries, etc., and provides end-to-end user problem solutions.

[0168] In addition, the customer service robot of the telecommunications customer service intelligent interaction system can understand user problems like humans, accurately match preset questions in the question-answer library from a semantic level, and complete user responses through service skill calls. It builds a telecommunications business knowledge base through data synchronization, conversion, data segmentation, index construction and other technologies, automates the knowledge base construction process, updates knowledge regularly, and can complete knowledge positioning and retrieval from massive knowledge bases with one click, and accurately return relevant knowledge fragments. With the help of retrieval enhancement generation technology, the customer service big model can quickly learn business knowledge, allowing the big model to acquire telecommunications field knowledge without training and fine-tuning, thereby preventing the potential risk of replying to users with expired product or business information, leading to user complaints.

[0169] Based on the reasoning and generation capabilities of the big model, the system can simulate the role of customer service personnel and automatically generate answers with professional customer service scripts, thereby effectively helping the business to quickly respond to user needs.

[0170] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0171] Refer to the following Figure 8 To describe a customer service interaction device 800 based on a large language model according to an embodiment of the present disclosure. Figure 8 The customer service interaction device 800 based on a large language model shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0172] The customer service interaction device 800 based on the large language model is expressed in the form of a hardware module or a software module. The components of the customer service interaction device 800 based on the large language model may include but are not limited to: an identification module 802, which is used to identify the intent scenario of the received user question; a determination module 804, which is used to identify that the user question belongs to a specified intent scenario and determine the corresponding domain knowledge base that matches the user question; a retrieval module 806, which is used to perform enhanced retrieval of the corresponding domain knowledge base based on the user question to obtain enhanced business knowledge fragments for the user question; a reply module 806, which is used to input the business knowledge fragments into the large language model so that the large language model performs filtered question reply processing based on the business knowledge fragments, and outputs the reply information of the user question based on the processing result.

[0173] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0174] The storage unit stores a program code, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform the following steps: Figure 2 The scheme described in .

[0175] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0176] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0177] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0178] The electronic device 900 may also communicate with one or more external devices 970 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 900, and / or any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 960. Fig. 9 As shown, the network adapter 960 communicates with other modules of the electronic device 900 via the bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0179] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or an electronic device, etc.) to execute the method according to the implementation of the present disclosure.

[0180] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to enable the electronic device to perform the steps according to various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0181] According to the program product for implementing the above method according to the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0182] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0183] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0184] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0185] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the IoT terminal, as an independent software package, partially on the user computing device and partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0186] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units to be concretized. In addition, although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.

[0187] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions so that a computing device (which can be a personal computer, a server, a mobile terminal, or an electronic device, etc.) executes the method according to the embodiment of the present disclosure. After considering the disclosure of the specification and practicing the disclosure here, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes the common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are indicated by the attached claims.

Claims

1. A customer service interaction method based on a large language model, characterized in that: include: Identify the intent scenarios of received user questions; Identify that the user question belongs to a specified intent scenario, and determine a corresponding domain knowledge base matching the user question; Performing enhanced retrieval on the corresponding domain knowledge base based on the user question to obtain enhanced business knowledge fragments for the user question; The business knowledge fragment is input into the large language model so that the large language model performs a filtering question answering process based on the business knowledge fragment, and outputs answer information of the user question based on the processing result.

2. The customer service interaction method based on a large language model according to claim 1, characterized in that: The business knowledge fragment includes a business background title and business background content, and the large language model includes a pre-filtering model, a post-filtering model, and a main language model. The business knowledge fragment is input into the large language model, so that the large language model performs a filtering question answering process based on the business knowledge fragment, and outputs the answer information of the user question based on the processing result, including: Inputting the business background title and the user question into the pre-filtering model to detect the subject consistency between the business background title and the user question based on the pre-filtering model; If the topics are consistent, the user question and the business background content are input into the main language model to generate information to be answered; Inputting the information to be answered and the user question into the post-filtering model to detect the correlation between the information to be answered and the user question based on the post-filtering model; If it is determined to be relevant, the information to be replied is determined as the reply information and fed back to the user end.

3. The customer service interaction method based on a large language model according to claim 1, characterized in that: Before identifying the intent scenario of the received user question, it includes: Based on the user identifier of the user question, detect whether the user's session history question-answer pair is empty to determine whether the received user question belongs to a multi-round question or a first-round question. If the session history question-answer pair is empty, it is determined that the user question belongs to the first-round question; if the session history question-answer pair is not empty, it is determined that the user question belongs to the multi-round question.

4. The customer service interaction method based on a large language model according to claim 1, characterized in that: Identifying that the user question belongs to a specified intent scenario and determining a corresponding domain knowledge base matching the user question includes: Determining that the received user question belongs to a multi-round question, the user question belongs to the specified intention scenario, and determining a specified historical question in the multi-round question that belongs to the specified intention scenario; The knowledge base consulted when querying and answering the specified historical question is used as the corresponding field knowledge base, Among them, the historical question among the multiple rounds of questions that has the shortest time interval with the user question and belongs to the designated intention scenario is determined as the designated historical question.

5. The customer service interaction method based on a large language model according to claim 4, characterized in that: Based on the user question, the corresponding domain knowledge base is searched in an enhanced manner to obtain a business knowledge fragment enhanced for the user question, including: Generate information to be retrieved based on the name of the corresponding domain knowledge base, the user question and the specified historical question; The corresponding domain knowledge base is searched based on the information to be retrieved to obtain the business knowledge fragment enhanced for the user question.

6. The customer service interaction method based on a large language model according to claim 5, characterized in that: Generating information to be retrieved based on the name of the corresponding domain knowledge base, the user question and the specified historical question includes: Obtaining the intermediate historical questions input between the user question and the designated historical question; The user question, the intermediate historical question and the name of the corresponding domain knowledge base are input into a query rewriting model for rewriting to obtain the information to be retrieved.

7. The customer service interaction method based on a large language model according to claim 1, characterized in that: Identifying that the user question belongs to a specified intent scenario and determining a corresponding domain knowledge base matching the user question also includes: It is determined that the user question is a first-round question, and it is identified that the first-round question belongs to the specified intention scenario, and the candidate knowledge bases are traversed to select the corresponding domain knowledge base that matches the domain of the first-round question.

8. The customer service interaction method based on a large language model according to claim 7, characterized in that: Based on the user question, the corresponding domain knowledge base is searched in an enhanced manner to obtain a business knowledge fragment enhanced for the user question, including: Retrieving the corresponding domain knowledge base based on the first round of questions to obtain context information enhanced for the user questions; The context information and the first round of questions are concatenated to serve as the enhanced business knowledge fragment.

9. The customer service interaction method based on a large language model according to claim 8, characterized in that: The corresponding domain knowledge base is searched based on the first round of questions to obtain context information enhanced for the user questions, including: Rewrite the query rewriting model for the first round of questions to obtain a rewritten question; Inputting the rewritten question into a question-answer pair library for matching; If they match, the reply information is generated; If there is no match, the corresponding domain knowledge base is searched based on the rewritten question to obtain the context information enhanced by the user question.

10. The customer service interaction method based on a large language model according to claim 7, characterized in that: Identify the intent scenarios of received user questions, including: If it is identified that the first round of questions do not belong to the specified intention scenario, locate the question intent to which the first round of questions belong; Determine a fixed script library configured for the question intention based on the positioning result; The answer information matching the first round of questions is searched from the fixed script library.

11. The customer service interaction method based on a large language model according to claim 10, characterized in that: Also includes: If it is determined that the received user question belongs to a multi-round question and the user question does not belong to the specified intent scenario, the user question is processed as the first-round question that does not belong to the specified intent scenario.

12. The customer service interaction method based on a large language model according to claim 3, characterized in that: Before detecting whether the user's session history question-answer pair is empty based on the user identifier of the user question, the method further includes: In response to the received user question, the robot performs answer semantic matching on the user question based on the robot knowledge base, If there is a matching result, the reply information is generated; if there is no matching result, whether the user's conversation history question and answer pair is empty is detected.

13. The customer service interaction method based on a large language model according to claim 6 or 9, characterized in that: Also includes: The query rewriting model is rewritten based on at least one of similar word expansion, knowledge graph association and similar semantic retrieval data sub-models.

14. The customer service interaction method based on a large language model according to claim 1, characterized in that: Before identifying the intent scenario of the received user question, it also includes: In response to receiving the original question information of the user, if it is detected that the original question information includes a data table, normalizing the data table to obtain a normalized table, wherein the normalized table has a first number of rows and a second number of columns; Decomposing and semantically converting the normalized table based on the row dimension and the column dimension to obtain the user question; If the original question information does not include the data table, the original question information is determined to be the user question.

15. The customer service interaction method based on a large language model according to claim 1, characterized in that: Identify the intent scenarios of received user questions, including: Based on the intention scenario recognition model, the intention scenario of the user question is recognized, and the probability distribution of different intention scenario categories is output; If the specified intention scenario has the highest probability in the probability distribution, it is determined to belong to the specified intention scenario, Among them, the designated scenario intention includes consultation type, and the non-designated scenario intention includes at least one of inquiry type, processing type, fault reporting type and complaint type.

16. The customer service interaction method based on a large language model according to claim 1, characterized in that: Before identifying the intent scenario of the received user question, it also includes: Clean the historical question-and-answer data in the corresponding field to obtain processed data; Based on the key-value pair relationship, the processed data is processed into blocks to obtain data blocks, wherein the data blocks include a business background title and a business background content, wherein the business background title corresponds to a key and the business background content corresponds to a value; A knowledge base index is constructed based on the key, so as to generate the corresponding domain knowledge base based on the knowledge base index and the data blocks.

17. A customer service interaction device based on a large language model, characterized in that: include: The recognition module is used to identify the intent scenario of the received user questions; A determination module, used to identify that the user question belongs to a specified intent scenario, and determine a corresponding domain knowledge base matching the user question; A retrieval module, used to perform enhanced retrieval on the corresponding domain knowledge base based on the user question to obtain enhanced business knowledge fragments for the user question; The answering module is used to input the business knowledge fragment into the large language model so that the large language model performs a filtering question answering process based on the business knowledge fragment, and outputs answer information of the user question based on the processing result.

18. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the customer service interaction method based on a large language model as described in any one of claims 1 to 16 by executing the executable instructions.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the customer service interaction method based on a large language model described in any one of claims 1 to 16 is implemented.

20. A computer program product having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the customer service interaction method based on a large language model described in any one of claims 1 to 16 is implemented.

Images