Retrieval enhancement method, device, intelligent customer service system and medium based on large model

By constructing a retrieval enhancement method based on a large model and combining the recall model with the large model for training, the problem of inaccurate training samples in the intelligent customer service system was solved, the accuracy of responses to user questions was improved, and more efficient recall and response ranking were achieved.

CN118708705BActive Publication Date: 2024-11-22ZHIZHESIHAIBEIJINGTECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411195032.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-22
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

Existing intelligent customer service systems, when trained using manually labeled sample pairs, cannot fully learn the similarity between user questions, resulting in limited recall capabilities of the recall model. Furthermore, the in-batch contrastive learning loss function may mistakenly treat positive samples as negative samples, leading to insufficient model training and inaccurate output responses.

Method used

By constructing a retrieval enhancement method based on a large model, a combination of a recall model and a large model is used. The pre-trained recall model is used to match user questions with a standard question-and-answer database. Iterative training is performed using positive and negative sample pairs. The contrastive learning loss function with mask and the large model are used to analyze the output probability of standard question-and-answer information to obtain the response results for user questions.

Benefits of technology

It significantly improves the accuracy of the intelligent customer service system in responding to user inquiries. By training with sample pairs of user questions and similar user questions, combined with a masked contrastive learning loss function, the recall effect of the recall model is improved. Furthermore, the accuracy is ensured by reordering the response content through a large model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118708705B_ABST
    Figure CN118708705B_ABST
Patent Text Reader

Abstract

The present application relates to the field of information retrieval technology, and provides a retrieval enhancement method, device, intelligent customer service system and medium based on a large model. After obtaining the user question content and the corresponding user intent input by the current user, the method uses a pre-trained recall model to match the input user question content with the standard questions in the standard question and answer library, and obtains a preset number of standard question and answer information matching the user question content; the standard question and answer information includes matching standard questions and corresponding standard replies; based on the user intent, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information; based on the output probability of each standard question and answer information, the reply result of the user question content is obtained. This method improves the accuracy of intelligent customer service in replying to user questions by building an algorithm model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of information retrieval technology, and more specifically, to a retrieval enhancement method, device, intelligent customer service system and medium based on a large model. Background Art

[0002] The semantic matching problem between user questions and standard question libraries in search scenarios has undergone a technological evolution from unsupervised learning to supervised learning, and from traditional machine learning to deep learning.

[0003] The existing method is to use manually labeled (user questions, standard questions) sample pairs to train the vector representation recall model when training the intelligent customer service system; in the online stage, obtain the user question vector and the vectors in the standard question library, calculate the topK standard questions similar to the user question vector, and take the reply content corresponding to the most similar standard question as the reply result.

[0004] However, the inventors found that the above-mentioned training samples only used sample pairs of (user questions, standard questions), and were unable to fully learn the similarities between user questions, resulting in limited recall capabilities of the recall model. In addition, the contrastive learning loss function of In-Batch used in the training phase would mistakenly treat some positive samples as negative samples, making it impossible to fully train the model, resulting in inaccurate responses output by the trained intelligent customer service system. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a retrieval enhancement method, device, intelligent customer service system and medium based on a large model, which improves the accuracy of intelligent customer service in replying to user questions by building an algorithm model.

[0006] In a first aspect, a retrieval enhancement method based on a large model is provided, which is applied to an intelligent customer service system. The method may include:

[0007] Get the user question content and corresponding user intention entered by the current user;

[0008] A pre-trained recall model is used to match the input user question content with the standard questions in the standard question and answer library to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content stored in the customer service system and standard questions in the standard question and answer library; the standard question and answer information includes matching standard questions and corresponding standard replies;

[0009] Based on the user intention, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information;

[0010] Based on the output probability of each standard question and answer information, a reply result of the user question content is obtained.

[0011] In a possible implementation, the positive sample pair includes a sample pair consisting of any historical user question content having the same historical user intent as the historical user question content and the first standard question, and a sample pair consisting of two different historical user question contents having the same historical user intent;

[0012] The negative sample pairs include a sample pair consisting of any historical user question content having a different historical user intention from the historical user question content and a second standard question, and a sample pair consisting of two different historical user question contents having different historical user intentions.

[0013] In a possible implementation, the backbone network model of the recall model is a Bert-like architecture model or an improved LLM architecture dual-tower model.

[0014] In a possible implementation, the training process of the recall model includes:

[0015] Obtain the historical user question content and corresponding historical user intentions stored in the intelligent customer service system;

[0016] Based on historical user intentions and a standard question-answer library, constructing training sample pairs including the positive sample pairs and the negative sample pairs;

[0017] The In-Batch comparative learning method is adopted to input the training sample pairs into the target neural network model, and the preset loss function is used to perform convergence processing on the output results of the target neural network model until the preset convergence conditions are met to obtain a trained recall model.

[0018] In a possible implementation, the preset loss function is a masked contrastive learning loss function.

[0019] In a possible implementation, the recall model includes: an input layer, a model layer, a representation aggregation layer, a representation output layer, a normalization layer, a matching layer, and an output layer;

[0020] An input layer, used to input the user question content into the model layer at a first input port, and to input any standard question in a standard question-answering library into the model layer at a second input port;

[0021] A model layer, used to obtain a first representation vector of the user question content and a second representation vector of the standard question;

[0022] A representation aggregation layer, used to perform sequence dimension aggregation on the first representation vector and the second representation vector of the standard question respectively, to obtain a first sentence granularity representation vector and a second sentence granularity representation vector;

[0023] A representation output layer, used to output a first sentence granularity representation vector and a second sentence granularity representation vector after being aggregated by the sequence dimension;

[0024] A normalization layer, used to normalize the first sentence granularity representation vector and the second sentence granularity representation vector to obtain a first normalized representation vector and a second normalized representation vector;

[0025] A matching layer, used to match the first normalized representation vector and the second normalized representation vector using a preset matching algorithm to obtain a matching score;

[0026] The output layer is used to output a preset number of standard questions with the highest matching scores and the corresponding standard responses.

[0027] In a possible implementation, a reply result of the user question content is obtained based on the output probability of each standard question and answer information.

[0028] If there is any standard question and answer information whose output probability is the largest and greater than the probability threshold, the standard reply in the standard question and answer information is determined as the reply result of the user's question content;

[0029] Alternatively, if there is at least one standard question and answer information whose output probability is the largest and is not greater than the probability threshold, the standard replies in the at least one standard question and answer information are all determined as the reply results of the user question content.

[0030] In a second aspect, a retrieval enhancement device based on a large model is provided, which is applied to an intelligent customer service system, and the device may include:

[0031] An acquisition unit, used to acquire the user question content and the corresponding user intention input by the current user;

[0032] A matching unit is used to match the input user question content with the standard questions in the standard question and answer library using a pre-trained recall model to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content and the standard question and answer library stored in the customer service system; the standard question and answer information includes matched standard questions and corresponding standard replies;

[0033] An analysis unit, configured to analyze a preset number of standard question and answer information using a pre-trained large model based on the user intention, and obtain an output probability of each standard question and answer information;

[0034] The acquisition unit is further configured to acquire a response result of the user's question content based on the output probability of each standard question and answer information.

[0035] In a third aspect, an intelligent customer service system is provided, the system comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0036] Memory, used to store computer programs;

[0037] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0038] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0039] The retrieval enhancement method based on a large model provided in the embodiment of the present application uses a pre-trained recall model to match the input user question content with the standard questions in the standard question and answer library after obtaining the user question content input by the current user and the corresponding user intent, and obtains a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content stored in the customer service system and standard questions in the standard question and answer library; the standard question and answer information includes matching standard questions and corresponding standard replies; based on user intent, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information; based on the output probability of each standard question and answer information, the reply result of the user question content is obtained. This method improves the accuracy of intelligent customer service in replying to user questions by building an algorithm model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A schematic diagram of a flow chart of a retrieval enhancement method based on a large model provided in an embodiment of the present application;

[0042] Figure 2A schematic diagram of the structure of a recall model provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of the structure of another recall model provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of the structure of a retrieval enhancement device based on a large model provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of the structure of an intelligent customer service system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. Unless otherwise defined, the technical terms or scientific terms used in this application should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "including" and other similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connected", "coupled" or "connected" and other similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0047] Figure 1 The present invention provides a flowchart of a large model-based retrieval enhancement method, which can be applied to the situation of searching for questions and answers in an intelligent customer service system. The method can be executed by an intelligent customer service system, wherein the system can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes but is not limited to: a computer device. Figure 1 As shown, a retrieval enhancement method based on a large model provided by an embodiment of the present invention may include the following steps:

[0048] Step S110: Obtain the user question content and corresponding user intention input by the current user.

[0049] Receive the user question in voice / text form input by the current user to the intelligent customer service system, and parse the user question content of the user question;

[0050] Natural language processing technology is used to identify the intent of the user's question content and obtain the user's intent of the user's question content. For example, if the user's question content is "How can I get a refund today", the user's intent is "refund".

[0051] Step S120: Use a pre-trained recall model to match the input user question content with the standard questions in the standard question and answer library to obtain a preset number of standard question and answer information matching the user question content.

[0052] Before executing this step, you need to train a recall model, including:

[0053] Obtain the historical user question content and corresponding historical user intentions stored in the intelligent customer service system;

[0054] Based on historical user intentions and the standard question and answer database, training sample pairs including positive sample pairs and negative sample pairs are constructed; specifically, a sample pair consisting of any historical user question content with the same / similar historical user intention as the historical user question content and a first standard question, and a sample pair consisting of two different historical user question contents with the same / similar historical user intentions are used as positive sample pairs; a sample pair consisting of any historical user question content with different / dissimilar historical user intentions as the historical user question content and a second standard question, and a sample pair consisting of two different / dissimilar historical user question contents with different historical user intentions are used as negative sample pairs.

[0055] The batch processing (In-Batch) contrastive learning method is adopted to input the training sample pairs into the target neural network model, and the preset loss function is used to converge the results output by the target neural network model until the preset convergence conditions are met to obtain a trained recall model.

[0056] That is to say, the pre-trained recall model is obtained by iteratively training the target neural network model based on positive and negative sample pairs consisting of historical user question content stored in the customer service system and standard questions in the standard question and answer library.

[0057] Furthermore, the recall model may include: an input layer, a model layer, a representation aggregation layer, a representation output layer, a normalization layer, a matching layer, and an output layer;

[0058] An input layer, used to input user question content into the model layer at a first input port, and to input any standard question in a standard question-answering library into the model layer at a second input port;

[0059] The model layer is used to obtain a first representation vector of the user's question content and a second representation vector of the standard question;

[0060] A representation aggregation layer, used to perform sequence dimension aggregation on the first representation vector and the second representation vector of the standard question respectively, to obtain a first sentence granularity representation vector and a second sentence granularity representation vector;

[0061] A representation output layer, used to output a first sentence granularity representation vector and a second sentence granularity representation vector after being aggregated by the sequence dimension;

[0062] A normalization layer, used to normalize the first sentence granularity representation vector and the second sentence granularity representation vector to obtain a first normalized representation vector and a second normalized representation vector;

[0063] The matching layer is used to match the first normalized representation vector and the second normalized representation vector using a preset matching algorithm to obtain a matching score; the matching algorithm can be a similarity algorithm, such as Pearson correlation coefficient, Euclidean distance, cosine similarity, etc.

[0064] The output layer is used to output a preset number (such as top K) of standard questions with the highest matching scores (i.e., standard questions to be processed) and the corresponding standard replies (standard replies to the standard questions to be processed).

[0065] In some embodiments, the backbone network model of the recall model (located at the model layer) is a dual-tower structure, which can be a Bert-like architecture model (such as BGE bge-large-zh-v1.5) or an improved large language model (Large LanguageModel, LLM) architecture dual-tower model (Qwen1.5 model).

[0066] like Figure 2 As shown in FIG. 1 , the structural diagram of the recall model when the backbone network model is a Bert-like architecture model. Figure 3 As shown in FIG, the structural diagram of the recall model when the backbone network model is an improved LLM architecture model. Figure 2 compared to, Figure 3 A mapping layer (fully connected MLP layer) is added between the representation aggregation layer and the representation output layer.

[0067] Input layer: For ease of explanation, the input layer is used to input user questions and standard questions, represented by (Q, A) pairs. Q is the user question, and A is the standard question. During the model training process, A can also be a similar user question with the same user intent.

[0068] Model layer: The backbone network model has a dual-tower structure, with Q / q input on one side and A / a input on the other side. After the backbone network inputs Q / q or A / a, the output vector is the hidden layer vector, denoted as or , thus obtaining the token-sized representation vector. The dimension is , l is the length of the input sequence, and h is the dimension of the hidden layer.

[0069] Representation aggregation layer: The representation aggregation layer (AGGR) is used to represent the hidden layer vector or Aggregate in the sequence dimension to obtain a representation vector at the sentence granularity or There are several ways to aggregate:

[0070] 1) For the backbone network, we choose the Bert architecture. We directly adopt , that is, the representation vector of the first CLS token in the sequence is used as the sentence representation, The calculation of is the same.

[0071] 2) Select LLM architecture for the backbone network. , That is, the MAX_POOLING function and the MEAN_POOLING function are used to aggregate the hidden layer representation in the sequence dimension, the two pooling vectors are concatenated, and then mapped through the mapping layer (fully connected MLP layer) to obtain the sentence representation. The calculation of is the same.

[0072] Normalization layer: or The vector is normalized, and the optional normalization scheme is L2 Norm, and we get or .

[0073] Matching layer: Calculate the matching score, i.e. , where f() is a matching function, such as a similarity algorithm. One possible definition is: , τ is the temperature coefficient.

[0074] Furthermore, the preset loss function is the masked contrastive learning loss function (mask-InfoNCE), which can be expressed as:

[0075]

[0076] Among them, m(.) is the mask function, when hour, The value is 0, otherwise the value is 1e9. Among them, ID(.) represents the numbering function. When it is a standard question, the value is the number of the standard question (the number of the standard question in the standard question and answer database); when it is a similar question, the value is the number of the standard question corresponding to the similar question. The specific number value can be the actual number of the content in the business system, or it can be the modulus of the hash function.

[0077] That is to say, during the training process, the masked contrastive learning loss function uses the model's output data, label data, and mask data to jointly calculate the loss function.

[0078] The masked contrastive learning loss function means that other standard problems in the batch may be the same as the "standard problems" in the positive samples. It is unreasonable to directly treat other standard problems in the batch as negative samples. We avoid this problem by introducing a mask and setting the loss function of negative samples where other standard problems in the batch = standard problems to 0.

[0079] Returning to step S120, after using the pre-trained recall model to match the input user question content with the standard questions in the standard question and answer library, the matching scores of the user question content and different standard questions will be obtained; the standard question and answer information consisting of a preset number of standard questions with the highest matching scores and the corresponding standard replies will be determined as the preset number of standard question and answer information matching the user question content.

[0080] Step S130: Based on the user's intention, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information.

[0081] A Prompt is constructed by combining a preset number of standard questions and corresponding standard replies output by the recall model. The response results that meet the user's intention among the preset number of standard replies are used as labels. A preset number of classification task training samples are constructed, and the neural network model (such as the Qwen1.5 model) is trained to obtain a trained large model.

[0082] Input a preset number of standard question and answer information into the trained large model, and output the output probability of the standard reply in each category of standard question and answer information.

[0083] Step S140: based on the output probability of each standard question and answer information, obtain the reply result of the user's question content.

[0084] If there is any standard question and answer information whose output probability is the largest and greater than the probability threshold, the standard reply in the standard question and answer information is determined as the reply result of the user's question content.

[0085] Alternatively, if there is at least one standard question and answer information whose output probability is the largest and is not greater than the probability threshold, then the standard replies in the at least one standard question and answer information are all determined as the reply results of the user's question content.

[0086] In some embodiments, during the process of training the recall model, if the number of positive sample pairs and / or negative sample pairs is too small, it will affect the accuracy of the training model. Therefore, if the number of positive sample pairs or negative sample pairs is less than a preset number threshold, they belong to minority samples. At this time, the minority samples need to be expanded, including:

[0087] The preset selective interpolation SMOTE algorithm is used to classify minority class samples according to their neighbor distribution information (such as according to question content, user intent, etc.) to obtain target samples of different categories; and target samples of different categories are interpolated to obtain new samples corresponding to target samples of different categories.

[0088] In a specific implementation, obtaining target samples of different categories includes: calculating the minority class samples of the k nearest neighbors of the target minority class samples in the minority class samples to obtain a first nearest neighbor sample set, where k is an integer greater than 0; and calculating the samples of the k nearest neighbors of the target minority class samples in the entire training sample pair to obtain a second nearest neighbor sample set, where k is an integer greater than 0; wherein the target minority class sample is any minority class sample in the minority class samples; if the first nearest neighbor sample set and the second nearest neighbor sample set do not have the same sample, then the target minority class sample is determined to be a noise sample. If the first nearest neighbor sample set and the second nearest neighbor sample set have the same sample, then the target minority class sample is determined to be the target sample.

[0089] Afterwards, target samples of different categories are interpolated to obtain new samples corresponding to target samples of different categories, such as performing triangular interpolation or linear interpolation.

[0090] Another expansion method may be: using a fuzzy clustering algorithm to cluster minority class samples and majority class samples, and determine the cluster centers and cluster radii of different clustering classes; based on the cluster centers and cluster radii of different clustering classes, generating new samples, which may be new samples of the minority class or new samples of the majority class.

[0091] In some embodiments, after obtaining the training sample pairs, the extreme learning machine can also be trained based on the training samples. Unlike traditional single hidden layer feedforward neural networks (SLFNs), the extreme learning machine randomly assigns input weights and hidden layer biases, and does not need to adjust parameters like error back propagation in neural networks. The output weights of the extreme learning machine network model are directly determined by solving the linear model, so the training phase of the extreme learning machine is completed through only one iteration, and the training speed is extremely fast. The network structure of the extreme learning machine includes: input layer, hidden layer and output layer, and the connection between the input layer and the hidden layer is through the input weight The connection between the hidden layer and the output layer is established by the output weight Establish.

[0092] The retrieval enhancement method based on the large model provided in this application can significantly improve the recall effect of the recall model by using (user questions, standard questions) and (user questions, similar user questions) sample pairs as training samples and using a masked contrastive learning loss function. In addition, by feeding the results of the recall model into the large model and reordering the topK standard questions recalled by the recall model, the real response content can be further ranked in a higher position.

[0093] Corresponding to the above method, the embodiment of the present application also provides a retrieval enhancement device based on a large model, such as Figure 4 As shown, the device comprises:

[0094] An acquisition unit 410 is used to acquire the user question content and the corresponding user intention input by the current user;

[0095] The matching unit 420 is used to use a pre-trained recall model to match the input user question content with the standard questions in the standard question and answer library to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on the positive sample pairs and negative sample pairs consisting of the historical user question content and the standard question and answer library stored in the customer service system; the standard question and answer information includes the matched standard questions and the corresponding standard replies;

[0096] An analysis unit 430 is used to analyze a preset number of standard question and answer information using a pre-trained large model based on the user intention to obtain an output probability of each standard question and answer information;

[0097] The acquisition unit 410 is further configured to acquire a response result of the user's question content based on the output probability of each standard question and answer information.

[0098] The functions of each functional unit of the large model-based retrieval enhancement device provided in the above-mentioned embodiments of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in the large model-based retrieval enhancement device provided in the embodiments of the present application will not be repeated here.

[0099] The present application also provides an intelligent customer service system. Figure 5 As shown, it includes a processor 510 , a communication interface 520 , a memory 530 and a communication bus 540 , wherein the processor 510 , the communication interface 520 , and the memory 530 communicate with each other via the communication bus 540 .

[0100] A memory 530, for storing computer programs;

[0101] The processor 510 is used to execute the program stored in the memory 530 to implement the following steps:

[0102] Get the user question content and corresponding user intention entered by the current user;

[0103] A pre-trained recall model is used to match the input user question content with the standard questions in the standard question and answer library to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content stored in the customer service system and standard questions in the standard question and answer library; the standard question and answer information includes matching standard questions and corresponding standard replies;

[0104] Based on the user intention, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information;

[0105] Based on the output probability of each standard question and answer information, a reply result of the user question content is obtained.

[0106] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0107] The communication interface is used for communication between the above electronic device and other devices.

[0108] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0109] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0110] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 1 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0111] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer executes the large model-based retrieval enhancement method described in any of the above embodiments.

[0112] In another embodiment provided by the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the large model-based retrieval enhancement methods described in the above embodiments.

[0113] Those skilled in the art will appreciate that the embodiments in the present application can be provided as methods, systems, or computer program products. Therefore, the embodiments in the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments in the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0114] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0118] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application fall within the scope of the claims and their equivalents in the embodiments of the present application, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A retrieval enhancement method based on a large model, characterized in that: Applied in an intelligent customer service system, the method includes: Get the user question content and corresponding user intention entered by the current user; A pre-trained recall model is used to match the input user question content with the standard questions in the standard question and answer library to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content stored in the customer service system and standard questions in the standard question and answer library; the standard question and answer information includes matching standard questions and corresponding standard replies; Based on the user intention, a pre-trained large model is used to analyze a preset number of standard question and answer information to obtain the output probability of each standard question and answer information; Based on the output probability of each standard question and answer information, obtaining a reply result of the user's question content; The positive sample pairs include a sample pair consisting of any historical user question content having the same historical user intent as the historical user question content and the first standard question, and a sample pair consisting of two different historical user question contents having the same historical user intent; The negative sample pairs include a sample pair consisting of any historical user question content having a different historical user intention from the historical user question content and a second standard question, and a sample pair consisting of two different historical user question contents having different historical user intentions; The training process of the recall model includes: Obtain the historical user question content and corresponding historical user intentions stored in the intelligent customer service system; Based on historical user intentions and a standard question-answer library, constructing training sample pairs including the positive sample pairs and the negative sample pairs; Using the In-Batch comparative learning method, the training sample pairs are input into the target neural network model, and a preset loss function is used to perform convergence processing on the output of the target neural network model until the preset convergence condition is met, so as to obtain a trained recall model; The preset loss function is a masked contrastive learning loss function, which is expressed as: Where N is the number of standard questions and the number of user questions in the training sample; is the i-th standard problem, is the i-th user problem, is the jth standard problem, is the jth user question, f(.) is the matching function, τ is the temperature coefficient, m(.) is the mask function, when hour, The value is 0, otherwise the value is 1e9, where ID(.) represents the numbering function. When it is a standard question, the value is the number of the standard question, which is the number of the corresponding standard question in the standard question and answer database. When it is a similar question, the value is the number of the standard question corresponding to the similar question.

2. The method according to claim 1, characterized in that The backbone network model of the recall model is a Bert-like architecture model or an improved LLM architecture dual-tower model.

3. The method according to claim 1, characterized in that The recall model includes: an input layer, a model layer, a representation aggregation layer, a representation output layer, a normalization layer, a matching layer and an output layer; An input layer, used to input the user question content into the model layer at a first input port, and to input any standard question in a standard question-answering library into the model layer at a second input port; A model layer, used to obtain a first representation vector of the user question content and a second representation vector of the standard question; A representation aggregation layer, used to perform sequence dimension aggregation on the first representation vector and the second representation vector of the standard question respectively, to obtain a first sentence granularity representation vector and a second sentence granularity representation vector; A representation output layer, used to output a first sentence granularity representation vector and a second sentence granularity representation vector after being aggregated by the sequence dimension; A normalization layer, used to normalize the first sentence granularity representation vector and the second sentence granularity representation vector to obtain a first normalized representation vector and a second normalized representation vector; A matching layer, used to match the first normalized representation vector and the second normalized representation vector using a preset matching algorithm to obtain a matching score; The output layer is used to output a preset number of standard questions with the highest matching scores and the corresponding standard responses.

4. The method according to claim 1, characterized in that Based on the output probability of each standard question and answer information, obtaining a reply result of the user's question content; If there is any standard question and answer information whose output probability is the largest and greater than the probability threshold, the standard reply in the standard question and answer information is determined as the reply result of the user's question content; Alternatively, if there is at least one standard question and answer information whose output probability is the largest and is not greater than the probability threshold, the standard replies in the at least one standard question and answer information are all determined as the reply results of the user question content.

5. A retrieval enhancement device based on a large model, characterized in that: Applied in an intelligent customer service system, the device comprises: An acquisition unit, used to acquire the user question content and the corresponding user intention input by the current user; A matching unit is used to match the input user question content with the standard questions in the standard question and answer library using a pre-trained recall model to obtain a preset number of standard question and answer information matching the user question content; the recall model is obtained by iteratively training the target neural network model based on positive sample pairs and negative sample pairs consisting of historical user question content and the standard question and answer library stored in the customer service system; the standard question and answer information includes matched standard questions and corresponding standard replies; An analysis unit, configured to analyze a preset number of standard question and answer information using a pre-trained large model based on the user intention, and obtain an output probability of each standard question and answer information; The acquisition unit is further configured to acquire a response result of the user's question content based on the output probability of each standard question and answer information; Training unit for: Obtain the historical user question content and corresponding historical user intentions stored in the intelligent customer service system; Based on historical user intentions and a standard question-answer library, constructing training sample pairs including the positive sample pairs and the negative sample pairs; Using the In-Batch comparative learning method, the training sample pairs are input into the target neural network model, and a preset loss function is used to perform convergence processing on the output of the target neural network model until the preset convergence condition is met, so as to obtain a trained recall model; The preset loss function is a masked contrastive learning loss function, which is expressed as: Where N is the number of standard questions and the number of user questions in the training sample; is the i-th standard problem, is the i-th user problem, is the jth standard problem, is the jth user question, f(.) is the matching function, τ is the temperature coefficient, m(.) is the mask function, when hour, The value is 0, otherwise the value is 1e9, where ID(.) represents the numbering function. When it is a standard question, the value is the number of the standard question, which is the number of the corresponding standard question in the standard question and answer database. When it is a similar question, the value is the number of the standard question corresponding to the similar question.

6. An intelligent customer service system, characterized in that: The system includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-4 when executing a program stored in a memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.