Answer generation

Abstract

Embodiments of the present description provide an answer generation method and apparatus, a storage medium, and an electronic device. The answer generation method comprises: acquiring an original question inputted by a user, and acquiring, in a knowledge base, a plurality of similar questions associated with the original question; determining a standard question set on the basis of the plurality of similar questions, wherein the standard question set comprises standard questions respectively corresponding to the plurality of similar questions; using a plurality of models of different structures to respectively perform sorting on the standard questions in the standard question set in respect to similarity with the original question to obtain a sorting result corresponding to the plurality of models of different structures; and on the basis of the sorting result corresponding to the plurality of models of different structures, selecting, from the standard question set, a target standard question having the highest similarity with the original question, and taking a standard answer of the target standard question as an answer of the original question.

Description

Answer Generation Technical Field

[0001] The embodiments of this specification relate to the field of intelligent question-answering technology, and in particular to methods, devices, storage media, and electronic devices for generating answers. Background Art

[0002] Currently, in the field of question-and-answering (Q&A), such as in insurance, script retrieval and matching are crucial capabilities. Users ask questions to insurance concierges on the customer side, and the corresponding answers are displayed on the business-to-business platform to assist the concierge in answering their questions. Accurately retrieving relevant answers from massive user Q&A knowledge bases is crucial to improving user satisfaction. Therefore, accurately retrieving relevant answers from these vast knowledge bases has become a pressing technical challenge.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this specification, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The embodiments of this specification provide a method, device, storage medium, and electronic device for generating an answer, which can accurately retrieve answers that match user questions from a massive knowledge base.

[0005] Other features and advantages of the embodiments of the present specification will become apparent from the following detailed description, or may be learned in part from the practice of the present specification.

[0006] In a first aspect, a method for answer generation is provided, which includes: obtaining an original question input by a user, and obtaining multiple similar questions associated with the original question in a knowledge base; determining a set of standard questions based on the multiple similar questions; wherein the set of standard questions includes standard questions corresponding to the multiple similar questions respectively; using multiple models with different structures to sort the similarities between each standard question in the set of standard questions and the original question, and obtaining sorting results corresponding to multiple models with different structures; based on the sorting results corresponding to the multiple models with different structures, selecting a target standard question with the highest similarity to the original question from the set of standard questions, and using the standard answer to the target standard question as the answer to the original question.

[0007] In combination with the first aspect, in some possible implementations, the multiple models of different structures include a refined large model, which is a large model for implementing a multiple-choice discrimination task obtained by supervised fine-tuning of a preset base large model based on target domain data. The multiple-choice discrimination task is to select a standard question that is most similar to the original question from multiple standard questions input.

[0008] In combination with the first aspect, in some possible implementations, the use of multiple models with different structures to sort each standard question in the standard question set by similarity with the original question, and obtaining sorting results corresponding to the multiple models with different structures includes: constructing a prompt instruction based on the standard question set and the original question; wherein the prompt instruction is used to prompt the refined ranking large model as the target role, and with reference to the target domain knowledge and task prompts, select the standard question in the standard question set that is most similar to the original question; inputting the prompt instruction into the refined ranking large model to obtain the standard question in the standard question set output by the refined ranking large model that is most similar to the original question; determining the similarity between the remaining standard questions and the standard question in the standard question set that is most similar to the original question; wherein the remaining standard questions include the remaining standard questions in the standard question set except the standard question that is most similar to the original question; obtaining the sorting result corresponding to the refined ranking large model based on the similarity between the standard question in the standard question set that is most similar to the original question and the remaining standard questions and the standard question in the standard question set that is most similar to the original question.

[0009] In combination with the first aspect, in some possible implementations, the task prompt includes: streamlining each standard question in the standard question set and the original question, and setting numbers for each simplified standard question and original question, the prompt instruction is used to prompt the refined ranking model as the target role, and refer to the target domain knowledge and task prompts to select the standard question that is most similar to the simplified original question from the simplified standard questions, and output the number of the standard question that is most similar to the simplified original question; the prompt instruction is input into the refined ranking model to obtain the standard question that is most similar to the original question in the standard question set output by the refined ranking model, including: inputting the prompt instruction into the refined ranking model to obtain the number of the standard question that is most similar to the original question from the simplified standard questions output by the refined ranking model.

[0010] In combination with the first aspect, in some possible implementations, selecting a target standard question with the highest similarity to the original question from the set of standard questions based on the sorting results corresponding to the multiple models of different structures includes: determining weight parameters corresponding to the multiple models of different structures respectively; performing weighted processing on the sorting results corresponding to the multiple models of different structures according to the weight parameters corresponding to the multiple models of different structures to obtain a weighted sorting result; and selecting a target standard question with the highest similarity to the original question from the set of standard questions based on the weighted sorting result.

[0011] In combination with the first aspect, in some possible implementations, obtaining multiple similar questions associated with the original question in the knowledge base includes: searching the knowledge base through a multi-way recall method to obtain similar questions of the original question recalled based on the multi-way recall method; screening the similar questions recalled through the multi-way recall method to obtain multiple similar questions associated with the original question.

[0012] In combination with the first aspect, in some possible implementations, the retrieval is performed in the knowledge base through a multi-way recall method to obtain similar questions of the original question recalled based on the multi-way recall method, including: determining the product corresponding to the original question in the knowledge base; performing a multi-way recall among similar questions under the product to obtain similar questions of the original question recalled based on the multi-way recall method.

[0013] In combination with the first aspect, in some possible implementations, the multiple models with different structures include: a representational model and / or an interactive model.

[0014] In combination with the first aspect, in some possible implementations, the representation model includes one of the following or any combination thereof: a sentence embedding model using a twin BERT network, a sentence embedding model that optimizes cosine values, and a sentence embedding model of simple contrastive learning.

[0015] In a second aspect, a device for generating an answer is provided, which includes: an original question acquisition module for acquiring the original question input by the user; a similar question acquisition module, and acquiring multiple similar questions associated with the original question in the knowledge base; a standard question determination module, which determines a standard question set based on the multiple similar questions; wherein the standard question set includes standard questions corresponding to the multiple similar questions respectively; a ranking result acquisition module, which uses multiple models with different structures to sort the similarities between each standard question in the standard question set and the original question, and obtain ranking results corresponding to multiple models with different structures; an answer acquisition module, which selects a target standard question with the highest similarity to the original question from the standard question set based on the ranking results corresponding to the multiple models with different structures, and uses the standard answer to the target standard question as the answer to the original question.

[0016] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for generating an answer when executing the computer program.

[0017] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed on a computer or a processor, the computer or processor executes the method for generating the answer as described above.

[0018] According to another aspect of the present specification, a computer program product comprising instructions is provided. When the computer program product is run on a computer or a processor, the computer or the processor is caused to perform the above-mentioned answer generation method.

[0019] The answer generation method, device, storage medium, and electronic device provided in the embodiments of this specification have the following technical effects.

[0020] In the solution provided by the embodiments of this specification, by obtaining multiple similar questions of the original question input by the user in the knowledge base, and based on the multiple similar questions, including a standard question set of standard questions corresponding to the multiple similar questions, the selection range of the target standard question that is most similar to the original question is expanded, and the possibility of accurately retrieving the answer that meets the user's question is improved to a certain extent. By adopting multiple models with different structures, each standard question in the standard question set is sorted by similarity with the original question, and the sorting results corresponding to the multiple models with different structures are obtained, and based on the sorting results corresponding to the multiple models with different structures, the target standard question with the highest similarity to the original question is selected from the standard question set, and the standard answer to the target standard question is used as the answer to the original question, which is conducive to combining the advantages of models with different structures and selecting the target standard question with the highest similarity to the original question, thereby accurately retrieving the answer that meets the user's question, i.e., the original question, in a massive knowledge base.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are incorporated into and constitute a part of this specification, illustrate embodiments consistent with this specification, and together with the specification, are used to explain the principles of this specification. Obviously, the drawings described below are only some embodiments of this specification, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0023] FIG1 shows a flow chart of a method for generating an answer according to an embodiment of this specification.

[0024] FIG2 shows a schematic diagram of a knowledge base provided in an embodiment of this specification.

[0025] FIG3 shows a schematic diagram of the implementation process of step 102 to step 104 provided in the embodiment of this specification.

[0026] FIG4 is a schematic diagram showing an implementation principle of the method for generating an answer provided in an embodiment of this specification.

[0027] FIG5 shows a schematic structural diagram of an apparatus for generating an answer according to an embodiment of this specification.

[0028] FIG6 shows a schematic structural diagram of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of this specification more clear, the implementation methods of this specification will be further described in detail below with reference to the accompanying drawings.

[0030] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Instead, they are merely examples of devices and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this specification will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of this specification. However, those skilled in the art will appreciate that the technical solutions of this specification may be practiced while omitting one or more of the above-mentioned specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this specification.

[0032] In addition, the accompanying drawings are merely schematic illustrations of this specification and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0033] Currently, in the question-and-answer field, such as insurance Q&A systems, script retrieval and matching are the most critical capabilities. Users ask questions to insurance concierges on the consumer side, and the corresponding answers are displayed on the business-to-business platform to assist the concierge in answering their questions. Accurately retrieving the right answers from the massive user Q&A knowledge base is crucial to improving user satisfaction.

[0034] Insurance concierge script recommendation is a key capability in the business-to-business (B2B) context. It takes user questions, provides recommended answers, and assists concierges in answering user questions, thereby improving their efficiency. In the insurance concierge scenario, since users know their questions will be answered by a real concierge, their original questions are often personalized and divergent, with ambiguous semantics and a high number of long-tail questions. The same knowledge point can be asked in many different ways, placing higher demands on the concierge's ability to answer the original user's questions. Therefore, how to accurately retrieve answers that match user questions from a massive knowledge base to provide reference for insurance concierges and improve their efficiency has become a pressing technical challenge.

[0035] To solve the above technical problems, the embodiments of this specification provide a method, device, storage medium and electronic device for answer generation. By adopting multiple models with different structures, each standard question in the standard question set is sorted by similarity with the original question, and sorting results corresponding to multiple models with different structures are obtained; combining the sorting results corresponding to multiple models with different structures, the target standard question with the highest similarity to the original question is selected, and the standard answer to the target standard question is used as the answer to the original question. The strengths of multiple models with different structures are combined to improve the final sorting effect, so as to accurately retrieve answers that meet the user's questions from the massive knowledge base, provide reference for insurance managers, and improve the work efficiency of insurance managers.

[0036] The following first introduces an embodiment of the method for generating answers involved in the embodiments of this specification.

[0037] The answer generation method in the embodiments of this specification is applied to an electronic device, which may be a server. Specifically, the server may be a background server capable of performing functions such as data transmission and data processing. Specifically, the server may be an electronic device with data computing, storage, and network interaction functions; or it may be a software program running on the electronic device to provide support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server may be a single server, several servers, or a server cluster formed by several servers.

[0038] It should be noted that the methods of the embodiments of this specification can be performed by a single electronic device, such as a computer or server, or can be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the methods of the embodiments of this specification, and the multiple devices will interact with each other to complete the above-mentioned answer generation method.

[0039] The flowchart of the method for generating an answer in the embodiment of this specification can be seen in FIG1 , which includes the following steps.

[0040] Step 102: Obtain the original question input by the user, and obtain multiple similar questions associated with the original question in the knowledge base.

[0041] Step 104: Determine a standard question set based on the multiple similar questions; wherein the standard question set includes standard questions corresponding to the multiple similar questions respectively.

[0042] Step 106: using multiple models with different structures, sorting the similarities between each standard question in the standard question set and the original question, and obtaining sorting results corresponding to the multiple models with different structures.

[0043] Step 108: Based on the ranking results corresponding to the multiple models with different structures, a target standard question with the highest similarity to the original question is selected from the standard question set, and the standard answer to the target standard question is used as the answer to the original question.

[0044] In the embodiment shown in FIG1 , by obtaining multiple similar questions of the original question input by the user in the knowledge base, and based on the multiple similar questions, including a standard question set of standard questions corresponding to the multiple similar questions, the selection range of the target standard question that is most similar to the original question is expanded, and the possibility of accurately retrieving the answer that meets the user's question is improved to a certain extent. By using multiple models with different structures, each standard question in the standard question set is sorted by similarity with the original question, and the sorting results corresponding to the multiple models with different structures are obtained, and based on the sorting results corresponding to the multiple models with different structures, the target standard question with the highest similarity to the original question is selected from the standard question set, and the standard answer to the target standard question is used as the answer to the original question. This is conducive to combining the advantages of models with different structures and selecting the target standard question with the highest similarity to the original question, thereby accurately retrieving the answer that meets the user's question, i.e., the original question, in the massive knowledge base.

[0045] The following describes the specific implementation of each step in the embodiment shown in FIG1 by taking the application of the answer generation method to the answer generation platform as an example.

[0046] In step 102, the user enters the original question via a mobile phone, computer, or other terminal device, which then sends the original question to the answer generation platform. The answer generation platform retrieves multiple similar questions from the knowledge base. These similar questions are semantically similar to the original question, but differ in how they are asked, i.e., in how they are expressed.

[0047] The knowledge base includes multiple knowledge points. Each knowledge point includes a standard question and multiple similar questions associated with the standard question. The multiple similar questions are also extended questions. The standard question and its associated similar questions / extended questions can be considered to be semantically similar, and the standard question and other standard questions can be considered to be semantically dissimilar.

[0048] Exemplarily, the knowledge base adopts a three-level structure of products, standard questions, and similar questions in its structural design. A knowledge point under the same product corresponds to a standard question, but may correspond to multiple similar questions. These multiple similar questions can be different expressions of the same knowledge point. For example, a schematic diagram of the knowledge base can be found in Figure 2, including: Product 1, Product 2...Product N. The knowledge points under Product 1 include: Knowledge Point 1, Knowledge Point 2...Knowledge Point N. The standard question corresponding to Knowledge Point 1 is Standard Question 1, the standard question corresponding to Knowledge Point 2 is Standard Question 2, and the standard question corresponding to Knowledge Point 3 is Standard Question 3. Similar questions associated with Standard Question 1 include: Similar Question 1, Similar Question 2...Similar Question N. Based on the knowledge base in Figure 2, after obtaining the original question input by the user, the answer generation platform can identify the product slot of the original question to determine the product corresponding to the original question, and then determine the knowledge point of the original question under the product, recall multiple similar questions similar to the original question and the same knowledge points in the knowledge base, and obtain multiple similar questions associated with the original question.

[0049] Exemplarily, the above-mentioned obtaining multiple similar questions associated with the original question in the knowledge base includes: searching in the knowledge base through a multi-way recall method to obtain similar questions of the original question recalled based on the multi-way recall method; screening among the similar questions recalled through the multi-way recall method to obtain multiple similar questions associated with the original question.

[0050] Among them, the multi-way recall method may include BM25 (Best Matching 25), vector recall, personalized configuration and other multi-way recall methods.

[0051] BM25 is a probability-based information retrieval model that is widely used in search engines, recommendation systems and other fields. Its core idea is to sort documents by calculating the similarity between documents and queries. The BM25 algorithm uses term frequency inverse document frequency (TF-IDF) to calculate the importance of each word in the document, and transfers the importance of the word to the query and document. The advantage of the BM25 algorithm is that it can take into account the distribution of words in the document, so as to better capture the semantic information between the document and the query. In this embodiment, based on the BM25 algorithm, the similarity between each similar question and the original question is calculated to sort each similar question, and the 20 similar questions ranked in the top 20 are taken out as similar questions to the original question recalled by the one-way recall method in the multi-way recall method.

[0052] Vector Recall differs from BM25 in that it uses deep learning techniques to represent text as vectors and implements retrieval based on these vectors. A common approach involves encoding text using a pre-trained language model and representing it as vectors. These vectors capture the semantic information of the text, thereby better reflecting the similarity between documents.

[0053] For example, offline, a semantic vector is calculated and stored for each similar question. After obtaining the original question, the semantic vector of the original question is obtained through a vector model. Then, based on the semantic vector of each similar question and the semantic vector of the original question, the cosine similarity of the semantic vectors of the original question and the similar question is calculated. Based on the cosine similarity, the similar questions are sorted from large to small, and the top 20 similar questions are taken as the similar questions of the original question recalled by the single recall method in the multi-way recall method. It should be noted that this embodiment only uses the top 20 as an example. In specific implementations, several similar questions ranked at the top can be selected according to actual needs.

[0054] In addition to BM25 and vector recall, we also attempt to configure personalized recall methods based on intent for personalized questions, increasing the recall of similar questions for difficult-to-understand and long-tail knowledge points.

[0055] After obtaining similar questions recalled by the multi-channel recall method, the similar questions recalled by the multi-channel recall method can be sorted in descending order according to the similarity between the similar questions recalled by the multi-channel recall method and the original question, and the similar questions ranked in the top 30-50 are selected as the multiple similar questions finally determined to be associated with the original question.

[0056] Selecting top 30-50 helps to meet the requirements of recall rate and time consumption at the same time, that is, to ensure a high recall rate while not wasting process time.

[0057] It should be noted that this embodiment only takes the top 30-50 as an example. In a specific implementation, several similar questions ranked at the top can be selected according to actual needs.

[0058] In this embodiment, a multi-channel recall method is used to ensure that all questions are called upon, and questions that are similar to the original question and have the same knowledge points as the original question are not missed as much as possible, and similar questions related to the original question are avoided to ensure the accuracy and comprehensiveness of the recall.

[0059] In one possible implementation, the above-mentioned retrieval in the knowledge base through the multi-way recall method to obtain similar questions of the original question recalled based on the multi-way recall method includes: determining the product corresponding to the original question in the knowledge base; performing multi-way recall among similar questions under the product to obtain similar questions of the original question recalled based on the multi-way recall method.

[0060] Combined with Figure 2, assuming that the product corresponding to the original question is determined to be product 1 in the knowledge base, the scope of this recall can be locked to similar questions under product 1, and multiple recalls can be performed among similar questions under product 1 to obtain similar questions of the original question based on the multiple recall method.

[0061] In this embodiment, by limiting the recall scope to similar issues with the same product as the original issue, it is beneficial to improve the recall accuracy and comprehensiveness while increasing the recall rate.

[0062] In step 104, based on the multiple similar questions obtained in step 102, a standard question corresponding to each obtained similar question is determined. As shown in FIG2 , in the knowledge base, each standard question is associated with multiple similar questions. Accordingly, each similar question also has an associated standard question. Therefore, a standard question corresponding to each obtained similar question can be determined in the knowledge base.

[0063] For example, the standard questions corresponding to the multiple similar questions obtained in step 102 can be combined into a standard question set. Alternatively, if the total number of standard questions corresponding to the multiple similar questions obtained in step 102 is greater than a preset number threshold, similar questions with low similarity to the original question can be deleted from the multiple similar questions obtained in step 102, and then a standard question set can be formed based on the standard questions corresponding to the remaining similar questions after deletion, to ensure that the total number of standard questions in the final standard question set does not exceed the preset number threshold, so as to avoid affecting the subsequent processing speed. The preset number threshold can be set according to actual needs, for example, 20, 25, etc.

[0064] Exemplarily, the implementation process of the above steps 102 to 104 can refer to FIG3 , which includes the following.

[0065] First, the original question entered by the user is obtained and rewritten to obtain the rewritten original question. Rewriting is to complete, modify, and improve the semantics of the original question based on the original question and the previous context.

[0066] Then, through multi-way recall methods such as BM25 recall, vector recall, and personalized configuration, similar questions to the original question are recalled in the knowledge base, and the top 30-50 similar questions are screened.

[0067] Next, the top 20 standard questions are obtained based on the top 30-50 similar questions. These top 20 standard questions form the standard question set. Generally, the number of standard questions corresponding to the top 30-50 similar questions will not exceed 20. If this number exceeds 20, the least similar questions are removed based on their similarity to the original question, so that the total number of standard questions does not exceed 20.

[0068] Furthermore, the top 20 standard questions are passed to the refined ranking side for subsequent refined ranking processing. The top 20 standard questions are the standard questions in the above-mentioned set of standard questions. The refined ranking processing on the refined ranking side is the processing flow in steps 106 and 108.

[0069] It should be noted that the above top 30-50 and top 20 are only exemplary and are not specifically limited in this embodiment. For example, the top 20-40 similar questions can also be filtered, or the top 10 standard questions can be obtained.

[0070] For example, when rewriting the original question, the standard questions in the knowledge base can also be rewritten. For example, the same query rewriting model can be used to rewrite the user's original question and the standard questions in the knowledge base to achieve certain style unification and semantic integrity optimization of the original question and the standard questions in the knowledge base, further reducing the difficulty of subsequent matching between the original question and the standard questions, so that only the degree of similarity at the semantic level can be considered.

[0071] In step 106, a plurality of models with different structures are used to sort the similarities between the standard questions in the above-mentioned standard question set and the original questions, and the sorting results corresponding to the plurality of models with different structures are obtained.

[0072] Illustratively, the multiple models of different structures include: representational models and / or interactive models.

[0073] The representational model uses an encoder to encode sentence vectors for the two texts (i.e., the text of the original question and the text of the standard question), and then fuses the two vectors through a shallow classifier. The representational model focuses more on the construction of the text representation layer, and will convert the text into a unique overall representation vector at the text representation layer. Its idea is based on the twin network, and it will use a multi-layer neural network to extract the overall semantics of the text before matching. The text representation layer can use common fully connected neural networks, convolutional neural networks, recurrent neural networks, or models based on attention mechanisms. Currently, the most commonly used model is the Bert-like model; and there are many ways to calculate the interaction of the matching layer: using dot products, cosine matrices, Gaussian distances, fully connected neural networks, or similarity matrices, etc. In our scenario, the cosine matrix method can generally be used.

[0074] Exemplarily, the representational model includes one of the following or any combination thereof: a sentence embedding model using a twin BERT network (Sentence-Bidirectional Encoder Representation from Transformers, Sentence-BERT), a cosine value optimized sentence embedding model (Cosine Sentence, CoSENT), and a simple contrastive learning of sentence embeddings (Simple Contrastive Learning of Sentence Embeddings, SimCSE).

[0075] Sentence-BERT is a dual-tower model based on BERT. During the training phase, sentences A and B (for example, the original question is sentence A, and the standard question is sentence B) are fed into the same BERT-like model (parameter sharing). The sentence vector u for sentence A and the sentence vector v for sentence B are then obtained using the [CLS] vector or the average of all character vectors. These vectors are then concatenated with u, v, and the element-wise |uv|, and finally a 0-1 binary classification is performed using softmax. During the inference phase, sentences A and B are fed into the fine-tuned Sentence-BERT model. A pooling strategy is selected to obtain the sentence vectors. The cosine function is then used to calculate the similarity between the two sentences, thereby determining the similarity between the original question and each standard question in the standard question set. This results in a model with the Sentence-BERT structure. The standard questions in the standard question set are ranked by their similarity to the original question.

[0076] However, the Sentence-BERT model also suffers from issues such as misaligned training and prediction objectives, and difficulty in fine-tuning. To address these issues, we attempted to directly optimize the cosine values ​​of sentence pairs, ensuring that the similarity of any negative sample pair within a batch is smaller than that of any positive sample pair. This led to the construction of a Cosine Sentence (CoSENT) model, similar to Circle Loss, which partially improved performance. Combining this model with the CoSENT structure, we ranked each standard question in the standard question set by their similarity to the original question, yielding the results of a ranking algorithm that ranks each standard question in the standard question set by their similarity to the original question.

[0077] SimCSE uses the concept of contrastive learning. The core idea of ​​contrastive learning is to shorten the distance between related samples and extend the distance between unrelated samples in a certain feature space. It is also best to make the samples evenly distributed in the vector space. The main reason is that the more evenly the sample distribution is, the higher the information entropy is, and the data can be better utilized.

[0078] SimCSE can be divided into unsupervised SimCSE and supervised SimCSE. In unsupervised SimCSE, positive samples are generated by applying different dropout masks to the same sentence, generating two similar representations, while negative samples are in-batch negatives. Supervised SimCSE constructs positive and negative samples based on the NLI (Natural Language Inference) dataset. Positive samples are sentence pairs with entailment relationships, while negative samples are sentence pairs with contradiction relationships (hard-negatives) plus in-batch negatives. This embodiment focuses on supervised SimCSE training. Using the model trained on the NLI dataset as a foundation, further training can be performed using pair-wise triples (original sentence, similar sentence, dissimilar sentence) constructed from a high-quality annotated dataset. Combined with the trained SimCSE model, each standard question in the standard question set is ranked by similarity to the original question, resulting in a SimCSE-structured model. The resulting ranking results are then ranked by similarity to the original question.

[0079] The interactive model feeds two texts (the original question and the standard question) into the encoder, allowing them to exchange information during the encoding process before producing the final result. The interactive model differs from the representational model in that it abandons the encoding-first-then-matching approach. Instead, it matches the texts at the input layer, and then uses the matching results for subsequent modeling. The interactive model's concept is to first capture the matching information between the two texts, convert the word-to-word matching information into a grayscale image, and then perform subsequent modeling and abstraction to obtain a final correlation evaluation. The interactive layer primarily uses a structure, such as an attention mechanism, to interact with the two texts at different granularities. The matching results at each granularity are then aggregated to produce a feature matrix representing this information. Various attention methods can be employed, and different attention mechanisms can produce different effects. The subsequent representation layer is responsible for abstracting the resulting feature matrix, further abstracting the matching information between the two sentences.

[0080] For example, the interactive model can be the Enhanced LSTM for Natural Language Inference (ESIM) text matching model. In the matching layer, it defines a detailed interaction between the vector representation of each token obtained using cross-attention and the original token vector. ESIM consists of four parts: an input layer, an interaction layer, an aggregation layer, and a prediction layer. Its key feature is the interaction layer, which interacts and enhances the features of two sentences (such as a standard question and an original question) to form new features. Feature interaction is a cross-attention process that weights the features of the two texts based on their mutual attention. Feature enhancement specifically involves feature concatenation and calculating differences and dot products. The difference is calculated to determine the difference between the original question and the original question, facilitating similarity calculations. Combined with ESIM, each standard question in the standard question set is ranked by similarity to the original question, resulting in a model with the ESIM structure. This ranking results in a ranking of each standard question in the standard question set by similarity to the original question.

[0081] Exemplarily, multiple models with different structures include a refined large model, which is a large model obtained by supervised fine-tuning of a preset base large model based on target domain data for realizing a multiple-choice discrimination task. The multiple-choice discrimination task is to select a standard question that is most similar to the original question from multiple input standard questions.

[0082] Among them, the target domain data is the data of the target domain to which the refined ranking large model is applied. For example, when the target domain is the insurance field, the target domain data can be insurance field data. Moreover, when the target domain is the insurance field, the base large model can select a large model with a stronger ability to understand financial scenarios, so that the insurance field data can be used to fine-tune the above-mentioned large model with a stronger ability to understand financial scenarios to obtain a refined ranking large model. The refined ranking large model can be used to implement multiple-choice discrimination tasks, where the "multiple" in the multiple-choice refers to multiple standard questions, that is, each standard question in the above-mentioned standard question set, and the "multiple" in the multiple-choice refers to one standard question, that is, the standard question with the highest similarity to the original question among the multiple standard questions.

[0083] Exemplarily, when the plurality of models of different structures include a refined large model, the implementation of step 106 includes using the refined large model to sort the standard questions in the set of standard questions by their similarity to the original questions, thereby obtaining a ranking result corresponding to the refined large model. The implementation of obtaining the ranking result corresponding to the refined large model includes the following steps S11 to S14.

[0084] S11: Construct prompt instructions based on the standard question set and the original question.

[0085] Among them, the prompt instruction is used to prompt the refined large model as the target role, and refer to the target domain knowledge and task prompts to select the standard question that is most similar to the original question in the standard question set.

[0086] The target role is the role in the target domain, and the target domain knowledge is the knowledge points within the target domain. For example, if the target domain is insurance, the target role could be a senior insurance industry practitioner, and the target domain knowledge is insurance knowledge. Task prompts are instructions for completing the multiple-choice discrimination task, similar to how a large-scale refined ranking model completes the multiple-choice discrimination task and what steps are required to complete the task.

[0087] For example, the task prompt is: to solve in two steps: the first step is to simplify the original problem and the standard problems in the standard problem set; the second step is to use the results of the first step to select. In other words, select the standard problem in the simplified standard problem set that is most similar to the simplified original problem.

[0088] Among them, streamlining the original questions and the standard questions in the set of standard questions can be understood as: removing some qualifiers that do not affect the semantics and are irrelevant. For example, the question before simplification is "It seems that I can't buy it, right?", and the question after simplification is "Is it impossible to buy it?". The above-mentioned qualifiers that do not affect the semantics and are irrelevant may include, for example: "we", "us", "we", "we", "ok", "thank you", "well", etc. However, this embodiment does not make specific restrictions on this. By streamlining the original questions and the standard questions in the set of standard questions, it is beneficial for the refined ranking model to quickly complete the multiple-choice discrimination task and quickly obtain the sorting results corresponding to the refined ranking model.

[0089] For example, assuming the original question is: 'It seems I can't buy it now, right?', the standard questions in the standard question set include: ['What are the principles of health disclosure?', 'How long do I need to pay?', 'How long is the coverage period?', 'What type of insurance is this product?', 'What is this optional coverage? Do I need to purchase it separately? Or is it included in the product?', 'When does the policy take effect?', 'What does the coverage period mean?', 'What is the reason for not being able to buy?', 'How do I choose the coverage version?', 'Under what circumstances can I apply for a claim?']. Based on this, according to the standard question set and the original question, the prompt instruction prompt1 can be constructed as follows.

[0090] Role play: You are a senior insurance industry practitioner.

[0091] ICL (In-Context Learning): Insurance domain knowledge. The user's current question has the encyclopedia / insurance knowledge intent. "Can't buy" and "can't buy" have the same meaning.

[0092] Task Instructions: Refer to insurance domain knowledge and follow the task prompts to determine which standard question in the set of standard questions is most similar to the original question "It seems that I can't buy it anymore, right?" In other words, select the standard question with the highest similarity to the original question among multiple standard questions.

[0093] COT (Chain-of-thought, task prompt): It is required to be solved in two steps. The first step is to simplify the original problem and the standard problems in the standard problem set; the second step is to make a selection based on the results of the first step.

[0094] S12: Inputting a prompt instruction into the refined large model to obtain a standard question that is most similar to the original question in the set of standard questions output by the refined large model.

[0095] If the prompt instruction prompt1 is input into the refined sorting model, the refined sorting model can output the standard question in the standard question set that is most similar to the original question based on prompt1.

[0096] Taking into account the fact that there is an upper limit on the time it takes for the process from the user inputting the original question to the user being provided with a suitable answer, if there are some longer standard questions in the standard question set, that is, standard questions with more text, it is easy to cause the processing time of the fine ranking model to be longer, resulting in a timeout problem, that is, causing the entire process to take longer than the above-mentioned upper limit time set in advance. This is because the fine ranking model is essentially a generation task. If the fine ranking model believes that the standard question that is most similar to the original question is particularly long, its generation is particularly time-consuming. In order to optimize this problem, the task prompt can be further adjusted from generating the standard question that is most similar to the original question to generating the number of the standard question that is most similar to the original question, that is, the result generated by the fine ranking model is a number. Based on this, some task constraints can be added to the above prompt1, and the task prompt can be further adjusted.

[0097] Exemplarily, the task prompt includes: streamlining each standard question and original question in the standard question set, and assigning numbers to each of the streamlined standard questions and original questions; prompt instruction prompt2 is used to prompt the refined ranking model as the target role, and with reference to the target domain knowledge and task prompt, select the standard question that is most similar to the streamlined original question from the streamlined standard questions, and output the number of the standard question that is most similar to the streamlined original question. Correspondingly, the implementation method of the above S12 includes: inputting the prompt instruction into the refined ranking model to obtain the number of the standard question that is most similar to the original question from the streamlined standard questions output by the refined ranking model.

[0098] Assuming that the original question and each standard question in the standard question set are the same as those in the above example, in one possible implementation, the constructed prompt instruction prompt2 may be as follows.

[0099] Role play: You are a senior insurance industry practitioner.

[0100] ICL (In-Context Learning): Insurance domain knowledge. The user's current question has the encyclopedia / insurance knowledge intent. "Can't buy" and "can't buy" have the same meaning.

[0101] Task Instructions: Refer to insurance domain knowledge and follow the task prompts to determine which standard question in the set of standard questions is most similar to the original question "It seems that I can't buy it anymore, right?" In other words, select the standard question with the highest similarity to the original question among multiple standard questions.

[0102] COT (Chain-of-thought) task: This task requires a three-step solution. First, simplify the original question and the standard questions in the set. Second, use the results of the first step to make a selection. Third, assign a corresponding number to the sentence selected in the second step, i.e., output the number of the standard question that is most similar to the simplified original question.

[0103] Task constraint: Output the number of the standard problem that is most similar to the original problem.

[0104] After the prompt instruction prompt2 is input into the refined sorting model, the refined sorting model can refer to insurance domain knowledge and follow the task prompt to select one of the following numbered sentences with the meaning most similar to "It seems that I can no longer buy it, right?" and give the number corresponding to the most similar sentence, ['A. What are the principles of health disclosure', 'B. How long does it take to pay?', 'C. How long is the coverage period?', 'D. What type of insurance is this product', 'E. What is this optional coverage? Do I need to purchase it separately? Or is it included in the product?', 'F. When does the policy take effect?', 'G. What does the coverage period mean?', 'H. What is the reason for not being able to buy it?', 'I. How to choose the insured amount version', 'J. In what situations can I apply for a claim?']. Finally, the refined sorting model outputs the sentence (standard question) that is most similar to "It seems that I can no longer buy it, right?" (original question) as number: [gMASK]".

[0105] Because prompt2 adds the task constraint of outputting the number of the most similar standard question to the original question, while prompt1 requires outputting the complete sentence of the most similar standard question to the original question, prompt2 can reduce the time required to perform multiple-choice discrimination tasks using a large fine-sorting model compared to prompt1.

[0106] S13: Determine the similarity between the remaining standard questions and the standard question in the standard question set that is most similar to the original question.

[0107] Among them, the remaining standard questions include the remaining standard questions in the standard question set except the standard question that is most similar to the original question obtained in S12. For example, the standard question set includes: standard question 1, standard question 2, standard question 3, and standard question 4. Assuming that the standard question that is most similar to the original question obtained in S12 is standard question 2, the remaining standard questions include: standard question 1, standard question 3, and standard question 4. Therefore, in S13, the similarity between standard question 2 and standard question 1, the similarity between standard question 2 and standard question 3, and the similarity between standard question 2 and standard question 4 can be calculated respectively. Specifically, the similarity calculation method here can be the Jaccard similarity coefficient. For example, the Jaccard coefficient values ​​between the remaining standard questions and the standard question in the standard question set that is most similar to the original question can be calculated respectively. The larger the Jaccard coefficient value, the higher the similarity.

[0108] S14: Obtain a ranking result corresponding to the refined ranking model based on the similarity between the standard question in the standard question set that is most similar to the original question and the remaining standard questions and the standard question in the standard question set that is most similar to the original question.

[0109] Specifically, the standard question in the set of standard questions that is most similar to the original question has the highest similarity score. The similarity scores of the remaining standard questions are determined based on their similarity to the standard question with the highest similarity score. The greater the similarity, the higher the similarity score. Based on this, the similarity scores of each standard question in the set of standard questions can be obtained. The standard questions can then be sorted from high to low based on their similarity scores, and this sorting result is used as the sorting result corresponding to the refined ranking model.

[0110] For example, in the above example, the standard question obtained in S12 that is most similar to the original question is standard question 2. The remaining standard questions include standard question 1, standard question 3, and standard question 4. In this example, standard question 2 has the highest similarity score. Assuming that the similarity between standard question 2 and standard question 1 is 90%, the similarity between standard question 2 and standard question 3 is 92%, and the similarity between standard question 2 and standard question 4 is 93%, the ranking result corresponding to the refined ranking model is: standard question 2, standard question 4, standard question 3, standard question 1, in descending order of similarity to the original question.

[0111] In this embodiment, by applying the large model technology to the ranking of each standard question in the standard question set, it is helpful to further improve the accuracy of the ranking.

[0112] In step 108, the ranking results corresponding to multiple models with different structures are comprehensively considered. The target standard question with the highest similarity to the original question is selected from the set of standard questions, and the standard answer to the target standard question is used as the answer to the original question. Each standard question has a preset standard answer, and the answer to the target standard question is the answer that matches the user's original question.

[0113] Exemplarily, the implementation method of the above-mentioned step 108 includes: determining the weight parameters corresponding to the multiple models with different structures; performing weighted processing on the sorting results corresponding to the multiple models with different structures according to the weight parameters corresponding to the multiple models with different structures, to obtain the weighted sorting results; and selecting the target standard question with the highest similarity to the original question from the standard question set according to the weighted sorting results.

[0114] Specifically, the inventor of the present application has found through comparative analysis of models with different structures that some models are good at solving the problem of literal meaning matching, and some models are good at solving the problem that the literal meanings are not very similar, but the semantics are similar. Based on this, a kind of model that can gather the strengths of different structures is proposed in the present embodiment, and the scheme of accuracy similar to "union" is realized. A group of weight parameters with better accuracy are searched by optimization method, and the weight parameters corresponding to the models of the same structure are obtained respectively. With the weight parameters corresponding to the models of a plurality of different structures respectively, the sorting results corresponding to the models of a plurality of different structures are weighted, and a refined sorting integrated model is obtained, and its effect obtains a breakthrough improvement. This refined sorting integrated model can be understood as being used for the weight parameters corresponding to the models of a plurality of different structures respectively, and the sorting results corresponding to the models of a plurality of different structures are weighted, and the sorting results after weighted treatment are obtained, and the problem of target standard is output according to the sorting results after weighted treatment.

[0115] The above optimization methods include BFGS (Broyden-Fletcher-Goldfarb-Shanno, quasi-Newton method), CG (Conjugate gradient, conjugate gradient method), L-BFGS-B (Limited-memory Broyden-Fletcher-Goldfarb-Shanno, BFGS method with limits), Nelder-Mead (simplex method), Powell (conjugate direction method), etc. In this embodiment, the Powell (conjugate direction method) with better effect is selected, that is, the weight parameters corresponding to multiple models with different structures are obtained by the conjugate direction method.

[0116] In a possible implementation, the weight parameters corresponding to the models of multiple different structures may also be pre-set according to actual needs, which is not specifically limited in this embodiment.

[0117] For example, the multiple models with different structures include: a refined large model, Sentence-BERT, CoSENT, and SimCSE. Each of these four models has its own ranking result. Based on the weight parameters corresponding to these four models, the ranking results corresponding to these four models, i.e., the four ranking results, are weighted to obtain a single weighted ranking result. This weighted ranking result is the final ranking result for each standard question in the set of standard questions.

[0118] Exemplarily, the weighted ranking result can be understood as the ranking result of the weighted similarity between the standard questions and the original questions obtained by each of the four models. For any standard question in the standard question set, all four models can output the similarity between the standard question and the original question. Based on the weight parameters corresponding to the four models, the similarity between the standard question and the original question output by the four models is weighted to obtain the weighted similarity between the standard question and the original question. The higher the weighted similarity, the higher the weighted ranking result. The standard question ranked first in the weighted ranking result is the target standard question.

[0119] In specific implementations, the weight parameters corresponding to models of different structures can be periodically adjusted according to different intentions to optimize the sorting effect.

[0120] In one possible implementation, the answer generation method flowchart is shown in Figure 4. It includes obtaining the original question, then identifying the product slot of the original question based on the product index, matching it with the knowledge base, and then entering the search and matching recall phase. Through a multi-way recall process, 30-50 similar questions (the top 30-50 similar questions) related to the original question are retrieved from the knowledge base. Next, the standard questions corresponding to each of the top 30-50 similar questions are determined, resulting in the top 20 standard questions. These top 20 standard questions then enter the refined ranking phase. In this refined ranking phase, the top 20 standard questions are ranked by relevance to the original question using multiple models with different structures (for example, the large refined ranking model, SimCSE model, and CoSENT model shown in Figure 4). The ranking results from these models are then weighted using an integrated refined ranking model. Finally, the top target standard question is output, and the corresponding answer is retrieved and recommended to the steward. This approach optimizes the accuracy of the target standard question obtained in the refined ranking phase, thereby improving the end-to-end accuracy and steward adoption rate of the entire process.

[0121] For example, in addition to using the aforementioned models with different structures, statistical features, such as various user behavioral characteristics, corresponding intent information, and labels, can also be added to train an XGBoost tree model to rank the standard questions in the standard question set. In addition to combining the ranking results corresponding to the models with different structures, the refined ranking ensemble model can also combine the ranking results corresponding to the XGBoost tree model, fully utilizing information from various aspects to achieve better ranking results.

[0122] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0123] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of this specification and are not intended to be limiting. It should be understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it should be understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0124] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] The embodiment of this specification also provides an answer generation device 500, as shown in Figure 5, including: an original question acquisition module 501, used to obtain the original question input by the user; a similar question acquisition module 502, and obtains multiple similar questions associated with the original question in the knowledge base; a standard question determination module 503, which determines a standard question set based on the multiple similar questions; wherein the standard question set includes standard questions corresponding to the multiple similar questions respectively; a sorting result acquisition module 504, which is used to use multiple models with different structures to sort the similarities of each standard question in the standard question set with the original question, and obtain sorting results corresponding to multiple models with different structures; an answer acquisition module 505, which is used to select a target standard question with the highest similarity to the original question in the standard question set based on the sorting results corresponding to the multiple models with different structures, and use the standard answer of the target standard question as the answer to the original question.

[0126] In one possible implementation, the multiple models with different structures include a refined large model, which is a large model obtained by supervised fine-tuning of a preset base large model based on target domain data and is used to implement a multiple-choice discrimination task. The multiple-choice discrimination task is to select a standard question that is most similar to the original question from multiple standard questions input.

[0127] In one possible implementation, the ranking result acquisition module 504 is specifically used to: construct a prompt instruction based on the standard question set and the original question; wherein the prompt instruction is used to prompt the refined ranking large model as the target role, and refer to the target domain knowledge and task prompts to select the standard question that is most similar to the original question in the standard question set output by the refined ranking large model; input the prompt instruction into the refined ranking large model to obtain the standard question that is most similar to the original question in the standard question set output by the refined ranking large model; determine the similarity between the remaining standard questions and the standard question that is most similar to the original question in the standard question set; wherein the remaining standard questions include the remaining standard questions in the standard question set except the standard question that is most similar to the original question; according to the standard question that is most similar to the original question in the standard question set and the similarity between the remaining standard questions and the standard question that is most similar to the original question in the standard question set, obtain the ranking result corresponding to the refined ranking large model.

[0128] In one possible implementation, the task prompt includes: streamlining each standard question in the standard question set and the original question, and setting numbers for each streamlined standard question and original question, the prompt instruction is used to prompt the refined ranking model as the target role, and refer to the target domain knowledge and task prompts to select the standard question that is most similar to the streamlined original question from the streamlined standard questions, and output the number of the standard question that is most similar to the streamlined original question; the sorting result acquisition module is specifically used to: input the prompt instruction into the refined ranking model to obtain the number of the standard question that is most similar to the original question from the streamlined standard questions output by the refined ranking model.

[0129] In one possible implementation, the answer acquisition module is specifically used to determine the weight parameters corresponding to the multiple models of different structures; according to the weight parameters corresponding to the multiple models of different structures, the sorting results corresponding to the multiple models of different structures are weighted to obtain the weighted sorting results; according to the weighted sorting results, the target standard question with the highest similarity to the original question is selected from the standard question set.

[0130] In one possible implementation, the similar question acquisition module 502 is specifically used to search the knowledge base through a multi-way recall method to obtain similar questions of the original question recalled based on the multi-way recall method; and to screen the similar questions recalled through the multi-way recall method to obtain multiple similar questions associated with the original question.

[0131] In one possible implementation, the similar question acquisition module 502 is specifically used to determine the product corresponding to the original question in the knowledge base; perform multi-way recall among similar questions under the product, and obtain similar questions of the original question recalled based on the multi-way recall method.

[0132] In a possible implementation, the multiple models with different structures include: a representational model and / or an interactive model.

[0133] In one possible implementation, the representation model includes one of the following or any combination thereof: a sentence embedding model using a twin BERT network, a sentence embedding model that optimizes cosine values, and a sentence embedding model using simple contrastive learning.

[0134] It should be noted that the answer generation device provided in the above embodiment is merely an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the answer generation device provided in the above embodiment and the answer generation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0135] The embodiments of this specification also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of any of the above-mentioned methods are implemented.

[0136] FIG6 shows a structural diagram of an electronic device according to an embodiment of the present disclosure. Referring to FIG6 , the electronic device 600 includes a processor 601 and a memory 602 .

[0137] In the embodiments of this specification, the processor 601 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state; the coprocessor is a low-power processor for processing data in the standby state.

[0138] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices. In some embodiments of this specification, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is executed by the processor 601 to implement the method of the embodiments of this specification.

[0139] In some embodiments, the electronic device 600 further includes a peripheral device interface 603 and at least one peripheral device. The processor 601, memory 602, and peripheral device interface 603 may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 603 via a bus, signal lines, or circuit boards. Specifically, the peripheral device includes at least one of a display screen 604, a camera 605, and an audio circuit 606.

[0140] The peripheral device interface 603 can be used to connect at least one input / output (I / O)-related peripheral device to the processor 601 and the memory 602. In some embodiments of this specification, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments of this specification, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 can be implemented on separate chips or circuit boards. This embodiment of this specification is not specifically limited to this.

[0141] The display screen 604 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 604 is a touch screen display, the display screen 604 is also capable of collecting touch signals on or above the surface of the display screen 604. The touch signals can be input as control signals to the processor 601 for processing. In this case, the display screen 604 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments of this specification, there can be one display screen 604, provided on the front panel of the electronic device 600; in other embodiments of this specification, there can be at least two display screens 604, provided on different surfaces of the electronic device 600 or in a foldable design; in still other embodiments of this specification, the display screen 604 can be a flexible display screen, provided on a curved or foldable surface of the electronic device 600. The display screen 604 can even be provided in a non-rectangular, irregular shape, i.e., a special-shaped screen. The display screen 604 can be made of materials such as a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0142] The camera 605 is used to capture images or videos. Optionally, the camera 605 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the electronic device, and the rear camera is arranged on the back of the electronic device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and virtual reality (VR) shooting function or other fusion shooting functions. In some embodiments of the present specification, the camera 605 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0143] Audio circuit 606 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals for input to processor 601 for processing. For the purpose of stereo sound collection or noise reduction, multiple microphones may be provided, respectively, at different locations within electronic device 600. The microphone may also be an array microphone or an omnidirectional microphone.

[0144] Power supply 607 is used to power the various components of electronic device 600. Power supply 607 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 607 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0145] The electronic device structure block diagram shown in the embodiment of this specification does not constitute a limitation on the electronic device 600. The electronic device 600 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0146] In the description of this specification, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. For those skilled in the art, the specific meanings of the above terms in this specification can be understood according to specific circumstances. In addition, in the description of this specification, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0147] The embodiments of this specification also provide a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of the above embodiments. If the various components of the workflow engine are implemented as software functional units and sold or used as independent products, they may be stored in the computer-readable storage medium.

[0148] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of this specification is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media (eg, digital versatile discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0149] It should be noted that the foregoing description of this specification is based on specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] The above descriptions are merely specific embodiments of this specification, but the scope of protection of this specification is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this specification should be covered by the scope of protection of this specification. Therefore, equivalent modifications made according to the claims of this specification are still within the scope of protection of this specification.

Claims

1. A method for answer generation, wherein: The method comprises: Obtaining an original question input by a user, and obtaining multiple similar questions associated with the original question in a knowledge base; Determine a set of standard questions based on the multiple similar questions; wherein the set of standard questions includes standard questions corresponding to the multiple similar questions respectively; Using multiple models with different structures, sorting the similarities between each standard question in the set of standard questions and the original question, and obtaining sorting results corresponding to the multiple models with different structures; According to the ranking results corresponding to the multiple models with different structures, a target standard question with the highest similarity to the original question is selected from the standard question set, and the standard answer to the target standard question is used as the answer to the original question.

2. The method according to claim 1, wherein The multiple models with different structures include a refined large model, which is a large model obtained by supervised fine-tuning of a preset base large model based on target domain data for realizing a multiple-choice discrimination task. The multiple-choice discrimination task is to select a standard question that is most similar to the original question from multiple input standard questions.

3. The method according to claim 2, wherein: The plurality of models with different structures are used to sort the similarities between the standard questions in the standard question set and the original questions, respectively, to obtain sorting results corresponding to the plurality of models with different structures, including: Constructing a prompt instruction based on the set of standard questions and the original question; wherein the prompt instruction is used to prompt the refined large model as a target role, and referring to the target domain knowledge and task prompts, selecting the standard question in the set of standard questions that is most similar to the original question; Inputting the prompt instruction into the refined large model to obtain the standard question that is most similar to the original question in the set of standard questions output by the refined large model; Determining the similarity between the remaining standard questions and the standard question in the set of standard questions that is most similar to the original question; wherein the remaining standard questions include the remaining standard questions in the set of standard questions except the standard question that is most similar to the original question; The ranking result corresponding to the refined ranking model is obtained based on the standard question in the standard question set that is most similar to the original question and the similarity between the remaining standard questions and the standard question in the standard question set that is most similar to the original question.

4. The method according to claim 3, wherein: The task prompt includes: simplifying each standard question and the original question in the standard question set, and setting a number for each simplified standard question and original question, the prompt instruction is used to prompt the refined large model as a target role, and refer to the target domain knowledge and task prompt to select the standard question that is most similar to the simplified original question from the simplified standard questions, and output the number of the standard question that is most similar to the simplified original question; The step of inputting the prompt instruction into the refined sorting model to obtain the standard question that is most similar to the original question from the set of standard questions output by the refined sorting model includes: The prompt instruction is input into the refined large model to obtain the number of the standard question that is most similar to the original question among the simplified standard questions output by the refined large model.

5. The method according to claim 1, wherein The step of selecting, from the set of standard questions, a target standard question having the highest similarity to the original question based on the ranking results corresponding to the multiple models with different structures includes: Determining weight parameters corresponding to the multiple models with different structures respectively; performing weighted processing on the ranking results corresponding to the multiple models with different structures according to the weight parameters corresponding to the multiple models with different structures, to obtain a weighted ranking result; According to the ranking result after the weighted processing, a target standard question with the highest similarity to the original question is selected from the standard question set.

6. The method according to claim 1, wherein The step of obtaining multiple similar questions associated with the original question in the knowledge base includes: Searching the knowledge base in a multi-way recall manner to obtain similar questions to the original question recalled in the multi-way recall manner; Similar questions recalled in a multi-way recall manner are screened to obtain multiple similar questions associated with the original question.

7. The method according to claim 6, wherein: The searching in the knowledge base by the multi-way recall method to obtain similar questions to the original question recalled by the multi-way recall method includes: Determining a product corresponding to the original question in the knowledge base; A multi-way recall is performed on similar issues under the product to obtain similar issues of the original issue recalled based on the multi-way recall method.

8. The method according to any one of claims 1 to 7, wherein: The multiple models with different structures include: representational models and / or interactive models.

9. The method according to claim 8, wherein The representation model includes one of the following or any combination thereof: a sentence embedding model using a twin BERT network, a sentence embedding model with optimized cosine values, and a sentence embedding model with simple contrastive learning.

10. A device for generating an answer, wherein: The device comprises: The original question acquisition module is used to obtain the original question input by the user; A similar question acquisition module acquires multiple similar questions associated with the original question in the knowledge base; A standard question determination module is configured to determine a standard question set based on the plurality of similar questions; wherein the standard question set includes standard questions corresponding to the plurality of similar questions respectively; a ranking result acquisition module, configured to use a plurality of models with different structures to respectively rank the similarities between each standard question in the standard question set and the original question, and obtain ranking results corresponding to the plurality of models with different structures; An answer acquisition module is used to select a target standard question with the highest similarity to the original question from the standard question set according to the sorting results corresponding to the multiple models with different structures, and use the standard answer of the target standard question as the answer to the original question.

11. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer or a processor, the computer or the processor is caused to execute the method according to any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

Images