Business question and answer method, device and equipment based on large model

Abstract

The embodiment of the invention belongs to the technical field of information, and particularly relates to a business question and answer method, device and equipment based on a large model. According to the method, after preliminary question-answering processing is carried out based on an FAQ question-answering engine, if the matching accuracy of an answer and a user question is lower than a set threshold value, the user question is vectorized into a high-dimensional vector for representation through a large language model; then searching a most relevant document paragraph in the pre-vectorized document data; based on a retrieval result, dynamically generating an answer according with the user question by utilizing a large language model; and finally, the answers are post-processed and returned to the user. The problems that a traditional question answering system is weak in semantic understanding, insufficient in complex problem processing capacity, poor in expansibility, high in maintenance cost and the like are effectively solved, and the accuracy, the coverage rate and the usability of intelligent question answering in the professional field are remarkably improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the field of information technology, and in particular, to a business question answering method and device based on a large model and equipment. BACKGROUND

[0002] At present, in the bank operation work, the business service desk needs to handle all kinds of problems of the internal personnel about the information system, including data query, business operation, system error, etc. Because there are many internal systems in the bank and they are continuously updated, the traditional manual support method is low in efficiency and heavy in workload, so more and more banks introduce intelligent question answering systems to reduce the pressure on the service desk by automatically answering common questions, and only when the robot cannot solve the problem will it be handled by manual work.

[0003] The following two technical routes are adopted in the prior art: one is the method based on traditional retrieval and matching. This method first obtains the preliminary reference information of the user's question through network search, then obtains more candidate answers from the local database through text matching and vector matching, and finally uses a reordering model to generate the final reply by synthesizing multiple results. However, this method relies heavily on keyword matching and has weak semantic understanding ability, and is prone to return incorrect or irrelevant answers, and has poor processing effect on complex and professional multi-step problems. In addition, this type of system cannot automatically handle new problems that are not included, has poor scalability, needs to be manually maintained, has high maintenance cost, and also lacks the ability to understand the context of multi-round dialogue. The second is the method based on retrieval augmented generation (RAG). This method first constructs a knowledge base, converts the user query into a high-dimensional vector, and uses a vector database (such as Milvus) for semantic similarity retrieval to obtain a relevant document set, and then selects the final answer through a hybrid scoring mechanism. Although this type of system can automatically build on a large amount of text data and has certain semantic understanding and generation ability, it is not ideal for generating rules-based and structured problems (such as calculation type and standard process type problems), and it is difficult to meet the user's requirements for accuracy and consistency. At the same time, a system based entirely on RAG usually consumes a lot of computing resources and has significant response delays, and the controllability and consistency of the answer are also difficult to guarantee.

[0004] Therefore, there is an urgent need for a more efficient, reliable and easy-to-maintain intelligent question answering solution. SUMMARY

[0005] To solve the problems of weak semantic understanding ability, poor processing effect on complex and professional problems, insufficient extensibility, high maintenance cost, and difficulty in adapting to high accuracy requirements of bank business in the prior art, embodiments of the present specification provide a business question answering method, device and equipment based on a large model, which can meet the intelligent question answering needs of bank service desks for high reliability, high coverage rate and low maintenance cost by fusing a dual-path collaborative scheme of traditional FAQ engines and the capabilities of new-generation large language models, and simultaneously utilizing the high efficiency and accuracy of FAQ engines and the deep semantic understanding and generation capabilities of large language models.

[0006] To solve the above technical problems, a first aspect of the embodiments of the present specification provides a business question answering method based on a large model, which comprises, after preliminary question answering processing based on a FAQ question answering engine,

[0007] When the matching accuracy of the answer of the FAQ question answering engine and the user question is lower than a first threshold value, the user question is vectorized by a large language model to convert it into a vectorized question represented by a high-dimensional vector;

[0008] Retrieving the most relevant document paragraphs from the pre-vectorized document data, wherein the document data contains vectorized segments of system manuals and vectorized segments of knowledge bases;

[0009] Based on the retrieved at least one document paragraph, dynamically generating an answer conforming to the user question by using a large language model;

[0010] Generating a final answer according to the answer conforming to the user question and returning it to the user.

[0011] Further, the preliminary question answering processing based on the FAQ question answering engine further comprises,

[0012] Further, retrieving the most relevant document paragraphs from the pre-vectorized document data further comprises,

[0013] Segmenting the system manual and the knowledge base content, and assigning a unique identifier to each segment of content;

[0014] Converting each segment of content carrying the identifier into a vectorized document paragraph represented by a high-dimensional vector.

[0015] Further, retrieving the most relevant document paragraphs from the pre-vectorized document data further comprises,

[0016] Storing the vectorized document paragraphs in a vector database;

[0017] Based on an approximate nearest neighbor search algorithm, an efficient vector retrieval index is constructed for all data in the vector database.

[0018] Further, the user question is vectorized by a large language model, and the vectorized question converted into a high-dimensional vector representation further includes,

[0019] A pre-training language model based on the Transformer architecture is adopted to map the text input of the user question into a high-dimensional vector representation containing semantic information through semantic encoding capability.

[0020] Further, dynamically generating an answer conforming to the user question by a large language model further includes,

[0021] Determining whether the most relevant document paragraph has context information;

[0022] If the context information is available, the retrieval result and the context information are constructed into an answer conforming to the user question by natural language generation technology;

[0023] If the context information is not available, the document paragraph is constructed into an answer conforming to the user question by a large language model based on the Transformer architecture.

[0024] Further, if the context information is not available, further includes,

[0025] Calculating the similarity score between the vectorized question and the vectorized paragraph in the document database;

[0026] Based on the similarity score, one or more document paragraphs with the highest relevance are selected as target data;

[0027] Determining whether the highest similarity score of the target data is higher than a second preset threshold;

[0028] If not, it is determined that the retrieval fails, and a retrieval failure prompt information is generated and returned;

[0029] If yes, the target data is input into a large language model to generate the answer conforming to the user question.

[0030] Further, inputting the target data into a large language model to generate the answer conforming to the user question further includes,

[0031] Retrieving the top N candidate information in the target data based on vector similarity and the similarity of the vectorized question, N being a positive integer set;

[0032] The first N pieces of candidate information and the user question are added to a first prompt instruction, and submitted to a general large model to instruct the general large model to generate an answer that best matches the user question, thereby forming a final answer with the user question.

[0033] A second aspect of the embodiments of the present specification provides a business question answering device based on a large model, the device comprising:

[0034] An FAQ question answering engine module configured to perform preliminary question answering processing based on an FAQ question answering engine;

[0035] A vectorization unit configured to, when the matching accuracy of the answer of the FAQ question answering engine and the user question is lower than a first threshold, vectorize the user question through a large language model to convert the user question into a vectorized question represented by a high-dimensional vector;

[0036] A retrieval unit configured to retrieve a document paragraph most relevant to the vectorized question from pre-vectorized document data, the document data including vectorized segments of a system manual and vectorized segments of a knowledge base;

[0037] An answer generation unit configured to dynamically generate an answer conforming to the user question based on the at least one retrieved document paragraph using a large language model;

[0038] An answer returning unit configured to generate a final answer based on the answer conforming to the user question and return the final answer to the user.

[0039] A third aspect of the embodiments of the present specification provides a computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, performs the instructions of the method of any of the preceding embodiments.

[0040] A fourth aspect of the embodiments of the present specification provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor of a computer device, performs the instructions of the method of any of the preceding embodiments.

[0041] A fifth aspect of the embodiments of the present specification provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor of a computer device, performs the instructions of the method of any of the preceding embodiments.

[0042] For the purposes of the present specification, the following terms are defined as follows: To make the purposes, technical solutions and advantages of the embodiments of the present specification clearer, the present application will be described in detail below with reference to the drawings. The embodiments provided by the present application mainly include the following steps: first, when the matching accuracy of the answer returned by the FAQ question and answer engine and the user question is lower than a set threshold, inputting the user question to a large language model for vectorization processing, converting it to a high-dimensional vector representation, and converting the natural language question to a machine-understandable semantic vector, providing a basis for subsequent semantic retrieval; then, retrieving the most relevant document paragraphs from the pre-constructed vectorized document database, the document data including segmented and vectorized data of system manuals and knowledge bases, enhancing retrieval accuracy through semantic similarity matching, and ensuring that the obtained reference content is highly relevant to the user's intention; subsequently, based on one or more retrieved document paragraphs, using a large language model to dynamically generate an answer that meets the user's question, and using the generation and reasoning capabilities of the large language model to generate an accurate, coherent, and contextually appropriate result by fusing the retrieved content; finally, post-processing the generated answer and returning it to the user as the final answer, ensuring the completeness, readability, and user acceptance of the output result. Through the above-mentioned collaborative mechanism of fusion retrieval and generation, the embodiments of the present application achieve effective response to complex, professional, and unseen questions, and at the same time, using the rapid response of FAQ and the semantic understanding ability of the large model, meet the comprehensive needs of high accuracy, high coverage, and low maintenance cost in the bank business question and answer scene. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0044] Figure 1 The figure shows a system schematic diagram of the business question and answer method based on a large model according to an embodiment of the present specification;

[0045] Figure 2 The figure shows a flowchart of the business question and answer method based on a large model according to an embodiment of the present specification;

[0046] Figure 3 The figure shows a flowchart of retrieving the most relevant document paragraphs from the pre-vectorized document data according to an embodiment of the present specification;

[0047] Figure 4 The figure shows a flowchart of retrieving the most relevant document paragraphs from the pre-vectorized document data according to an embodiment of the present specification;

[0048] Figure 5 A flowchart of inputting the target data into a large language model to generate an answer consistent with a user question by an embodiment of the present specification is shown;

[0049] Figure 6 A schematic diagram of a large model-based business question and answer device according to an embodiment of the present specification is shown;

[0050] Figure 7 A structural diagram of a computer device according to an embodiment of the present specification is shown.

[0051] Explanation of the drawing symbols:

[0052] 101, terminal

[0053] 102, server

[0054] 601, FAQ question and answer engine module

[0055] 602, vectorization unit

[0056] 603, retrieval unit

[0057] 604, answer generation unit

[0058] 605, answer return unit

[0059] 702, computer device

[0060] 704, processor

[0061] 706, memory

[0062] 708, drive mechanism

[0063] 710, input / output module

[0064] 712, input device

[0065] 714, output device

[0066] 716, presentation device

[0067] 718, graphical user interface

[0068] 720, network interface

[0069] 722, communication link

[0070] 724, communication bus DETAILED DESCRIPTION

[0071] With reference to the drawings of the embodiments of the present specification, the technical solutions in the embodiments of the present specification will be described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present specification, rather than all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present specification.

[0072] It should be noted that the terms "first", "second" and the like in the description of the specification and claims of the present specification and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or equipment comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or equipment.

[0073] The present specification provides method operation steps as described in the embodiments or flowcharts, but can include more or fewer operation steps based on routine or non-creative work. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. In actual system or device product execution, the method order shown in the embodiments or drawings can be executed in sequence or in parallel.

[0074] It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solutions of the embodiments of the present specification comply with the relevant provisions of national laws and regulations.

[0075] It should be noted that in the embodiments of the present specification, some industry existing solutions of software, components, models, etc. may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the embodiments of the present specification, but does not mean that the applicant has or will necessarily use the solution.

[0076] As Figure 1The diagram illustrates a system implementation of a business question-and-answer method based on a large model, as described in this specification. The system may include a terminal 101 and a server 102. A communication connection is established between the terminal 101 and the server 102, enabling data interaction. The terminal 101 receives user-input questions and sends them to the server 102. Upon receiving the question, the server 102 processes it and generates an answer according to the question-and-answer method provided in the above embodiments of this specification, ultimately returning the result to the terminal 101. The terminal 101 displays the returned answer on an interface or stores it locally, and may further push it to relevant business personnel for review and subsequent operations.

[0077] In the embodiments of this specification, the server 102 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0078] It should be noted that, Figure 1 The example shown is merely one application environment provided by this disclosure. In practical applications, other application environments may also be included, which are not limited in the embodiments of this specification.

[0079] To address the problems existing in the prior art, this specification provides a method, apparatus, device, and storage medium for checking the end-to-end high availability capability of a business system. Figure 2 The diagram illustrates a business question-answering method based on a large model, as provided in an embodiment of this specification. It depicts a complete closed-loop process from data input to solution generation and feedback optimization. The order of steps listed in the embodiment is merely one possible execution order among many and does not represent the only possible order. In actual system or device products, the method can be executed sequentially or in parallel, as shown in the embodiment or the accompanying drawings. Specifically, as... Figure 2 As shown, the method may include: after performing preliminary question-and-answer processing based on the FAQ question-and-answer engine, it may also include,

[0080] Step 201: When the matching accuracy between the FAQ question-answering engine's answer and the user's question is lower than the first threshold, the user's question is vectorized using a large language model, and converted into a vectorized question with a high-dimensional vector representation.

[0081] Step 202: Retrieve the document paragraphs most relevant to the vectorization problem from the pre-vectorized document data, which includes vectorized segments of the system manual and vectorized segments of the knowledge base;

[0082] Step 203: Based on at least one of the retrieved document paragraphs, dynamically generate an answer that matches the user's question using a large language model;

[0083] Step 204: Generate a final answer based on the answer that matches the user's question and return it to the user.

[0084] In the embodiments of this specification, when the matching accuracy between the answer returned by the FAQ question-answering engine and the user's question is lower than a set threshold, the user's question is input into a large language model for vectorization processing, converting it into a high-dimensional vector representation. This transforms the natural language question into a machine-understandable semantic vector, providing a foundation for subsequent semantic retrieval. Next, the most relevant document paragraphs to the vectorized question are retrieved from a pre-built vectorized document database. The document data includes segmented and vectorized data from system manuals and knowledge bases. Semantic similarity matching enhances retrieval accuracy, ensuring that the obtained reference content is highly relevant to the user's intent. Subsequently, based on one or more retrieved document paragraphs, the large language model dynamically generates answers that match the user's question. Leveraging the generation and reasoning capabilities of the large language model, the retrieved content is integrated to generate accurate, coherent, and context-appropriate results. Finally, the generated answers are post-processed and returned to the user as the final answer, ensuring the completeness, readability, and acceptability of the output results. The embodiments of the present invention, through the above-mentioned collaborative mechanism of fusion retrieval and generation, achieve effective responses to complex, professional and unseen questions. At the same time, by utilizing the rapid response of FAQs and the semantic understanding capabilities of large models, it meets the comprehensive requirements of high accuracy, high coverage and low maintenance costs in banking business question-and-answer scenarios.

[0085] The initial question-and-answer processing based on the FAQ question-and-answer engine further includes the first-layer processing module of the system. This engine relies on traditional natural language processing algorithms to achieve question-and-answer matching, its core being efficient retrieval and answer return through a pre-built structured corpus. The corpus is organized using a combination of standard questions, corresponding standard answers, and multiple similar questions, and stored in a MySQL database. When a user asks a question, the system first uses a keyword-based recall algorithm (such as BM25) to recall several candidate standard questions or similar questions most similar to the user's question from the corpus. Subsequently, the system performs deduplication and mapping processing on the recall results, uniformly mapping similar questions to their corresponding standard questions to determine the best match. Each standard question may correspond to multiple similar questions; therefore, when uniformly mapping similar questions to their corresponding standard questions, multiple duplicate results will appear. The standard question with the highest frequency of duplication is selected as the best match. Finally, the system retrieves its preset standard answer based on the best match and returns it to the user as the optimal result, completing the initial automated question-and-answer process.

[0086] In one embodiment of this specification, to address the problem of insufficient accuracy and efficiency in document retrieval, such as Figure 3 As shown, retrieving the document paragraphs most relevant to the vectorization problem from the pre-vectorized document data further includes,

[0087] Step 301: Divide the system manual and the knowledge base content into segments and assign a unique identifier to each segment;

[0088] Step 302: Convert each segment of content carrying the identifier into a vectorized document segment represented by a high-dimensional vector.

[0089] Specifically, the system manual and knowledge base content are segmented, and each segment is assigned a unique identifier. By structurally segmenting long texts and assigning unique identifiers to each segment, subsequent precise segment-level retrieval and tracing are effectively supported, improving the granularity and flexibility of knowledge management. Subsequently, each segment carrying an identifier is converted into a vectorized document segment with a high-dimensional vector representation. This allows for the encoding of text semantics into high-dimensional vectors using a large language model (such as GPT), achieving a mathematical representation of deep text semantics and laying the foundation for efficient semantic retrieval based on vector similarity. To support efficient retrieval and generation capabilities, the system manual and knowledge base need to be pre-vectorized for storage. First, the system manual and knowledge base documents are segmented, and each segment is assigned a unique identifier, enabling fine-grained organization and management of the text. Then, using a large language model, each text segment is converted into a high-dimensional vector representation and stored in a dedicated vector database to support subsequent large-scale vector similarity retrieval.

[0090] In one embodiment of this specification, to solve the problem of efficient matching and retrieval of massive document paragraphs, such as Figure 4 As shown, retrieving the document paragraphs most relevant to the vectorization problem from the pre-vectorized document data further includes,

[0091] Step 401: Store the vectorized document paragraphs in a vector database;

[0092] Step 402: Based on the approximate nearest neighbor search algorithm, construct an efficient vector retrieval index for all data in the vector database.

[0093] Specifically, by storing semantic vectors in a database optimized for high-dimensional data, efficient storage and management of vectorized data is achieved, providing a foundation for subsequent rapid similarity retrieval. Subsequently, based on the approximate nearest neighbor (ANN) search algorithm, an efficient vector retrieval index is constructed for all data in the vector database. This technique significantly improves the retrieval speed and accuracy in large-scale vector sets by establishing an optimized index structure, ensuring that the system can locate the document paragraph most relevant to the user's question within milliseconds. In this embodiment, the generated vectorized document paragraphs are stored in a dedicated vector database to ensure data structure consistency and access efficiency. Furthermore, the system constructs an efficient retrieval index for all vector data based on the Approximate Nearest Neighbor (ANN) search algorithm. This index structure can significantly accelerate similarity calculation and matching operations during the query process, thereby ensuring that users can still obtain fast and accurate retrieval results even under high concurrency access.

[0094] In one embodiment of this specification, to address the problems of inaccurate semantic understanding of user questions and limited effectiveness of traditional keyword matching, the user question is vectorized using a large language model, transforming it into a vectorized problem with a high-dimensional vector representation. This further includes...

[0095] A pre-trained language model based on the Transformer architecture is used to map the text input of the user's question into a high-dimensional vector representation containing semantic information through semantic encoding capabilities.

[0096] Specifically, user questions are transformed into high-dimensional vector representations using a large language model. The pre-trained language model based on the Transformer architecture, leveraging its powerful semantic encoding capabilities, maps the user's input natural language text into high-dimensional vectors rich in semantic information. This achieves deep semantic understanding and representation of the user's question intent, effectively overcoming the reliance of traditional methods on superficial word order and literal matching. It significantly improves the accuracy and robustness of subsequent retrieval processes, providing a crucial foundation for the overall system to achieve accurate question answering.

[0097] In one embodiment of this specification, to address the compatibility issue between search results and answer generation, dynamically generating answers that match the user's question using a large language model further includes:

[0098] Determine whether the most relevant document paragraph has contextual information;

[0099] If contextual information is available, natural language generation technology is used to construct an answer that matches the user's question using the search results and the contextual information.

[0100] If no contextual information is available, the large language model of the Transformer architecture is used to construct the document paragraphs into an answer that matches the user's question.

[0101] Specifically, the system determines whether the most relevant document paragraphs possess contextual information. By assessing the completeness of the retrieved content, it provides a basis for subsequent generation strategies, ensuring the reliability and effectiveness of the answer construction. If contextual information is available, natural language generation technology is used to construct an answer that matches the user's question using the retrieval results and the contextual information. This fully integrates the retrieval content with relevant contextual information, leveraging the semantic combination and reasoning capabilities of the generative model to output an accurate, coherent, and user-intent-aligned response. If contextual information is lacking, a large language model based on the Transformer architecture is used to construct an answer that matches the user's question using the document paragraphs. This fully leverages the generalization generation advantages of the large language model, enabling the generation of structurally sound and content-relevant responses based on parameterized knowledge even when faced with incomplete document paragraphs, thereby improving the system's response coverage and robustness. In a specific embodiment, if the retrieved document provides sufficient contextual information, the system uses a Transformer-based generative model to integrate vectorized document content with natural language generation technology to construct an answer that fits the user's question. If the retrieved information is insufficient, the large model will be unable to generate a high-quality answer. In this case, the system will return a preset "retrieval failed" message, which will clearly indicate the generation status and reliability of the results while ensuring user experience.

[0102] In one embodiment of this specification, to address the issue of the system response mechanism when the search results are of poor quality, if contextual information is unavailable, matching the most relevant document paragraph in the document data further includes...

[0103] Calculate the similarity score between the vectorized problem and the vectorized paragraphs in the document database;

[0104] Based on the similarity score, one or more document paragraphs with the highest relevance are selected as target data;

[0105] Determine whether the highest similarity score of the target data is higher than a second preset threshold;

[0106] If not, the search is deemed to have failed, and a search failure message is generated and returned.

[0107] If so, the target data is input into the large language model to generate an answer that matches the user's question. Specifically, by setting a relevance threshold, strict quality control of the search results is implemented, effectively avoiding the risk of forcibly generating unreliable answers when information is insufficient or relevance is low, ensuring the accuracy and reliability of the system output. The mechanism of returning explicit prompts enhances the transparency of the system and the user-friendliness of the interaction, enabling users to clearly understand the current query status and adjust their questioning strategy in a timely manner. The system measures the relevance of the search results to the user's question by calculating vector similarity and sets a second preset threshold as the minimum quality requirement for generating answers. If the relevance scores of all candidate paragraphs do not reach this threshold, the search is deemed to have failed, the subsequent generation process is not triggered, and a preset "search failed" prompt is directly returned to the user. This function is implemented through decision logic integrated into the question-and-answer process, ensuring that the system can maintain predictability of behavior and reliability of results even in complex query scenarios.

[0108] In one embodiment of this specification, to address the issues of information organization and result reliability in scenarios with multiple answer outputs, such as... Figure 5 As shown, inputting the target data into a large language model to generate an answer that matches the user's question further includes,

[0109] Step 501: In the target data, retrieve the top N candidate information items with the highest similarity to the vectorized problem based on vector similarity, where N is a set positive integer;

[0110] Step 502: Add the top N candidate information and the user question to the first prompt instruction and submit it to the general big model to instruct the general big model to generate the answer that best matches the user question, and then combine it with the user question to form the final answer.

[0111] Specifically, in one embodiment of this specification, to address the issues of information organization and result reliability in multi-answer output scenarios, inputting the target data into a large language model to generate an answer that matches the user's question further includes the following steps: First, based on vector similarity, the top N candidate information items with the highest similarity to the vectorized question are retrieved from the target data, where N is a set positive integer. This step accurately filters out the information fragments most relevant to the user's question from massive amounts of data by calculating the similarity between semantic vectors, providing high-quality context for answer generation. Subsequently, the top N candidate information items and the user's question are added to a first prompt instruction and submitted to the general large model. This operation organically combines the retrieval results with the user's question by constructing a prompt instruction, providing the general large model with sufficient contextual background and clear generation guidance, ultimately generating the answer that best matches the user's question, thereby forming an accurate, reliable final answer that highly matches the user's intent.

[0112] In another embodiment of this specification, in order to continuously optimize the question-answering capabilities of the large language model, it is also possible to...

[0113] The initial large model is iteratively trained using the final answer. In each training round, the judge model scores the final answer to the user question generated by the initial large model and the preset answer to the similar question in the corpus.

[0114] Based on the scoring results, answers that meet the preference requirements are selected and combined with the domain questions to form target training data. The initial large model is trained using the target training data, and a question-answering large language model is obtained after the iterative training.

[0115] This method continuously optimizes an initial large-scale language model using the final answers generated by the system. The specific training process includes: In each iteration, a pre-trained referee model independently scores two sets of answers: one set is the final answer generated by the initial large-scale model for a user question, and the other set is preset standard answers corresponding to similar questions from a standard corpus. The referee model quantifies the scores based on dimensions such as accuracy, relevance, and completeness, and selects high-quality answers that meet preset preference requirements based on the scoring results. Subsequently, the selected high-quality answers are combined with corresponding domain questions to form target training data, which includes both high-quality answers generated by the system and manually labeled standard answers. Finally, the initial large-scale model is subjected to supervised fine-tuning training using the constructed target training data. Through multiple rounds of iterative training and parameter optimization, a question-answering large-scale language model with significantly improved performance is obtained. This model maintains its generative capabilities while significantly improving the accuracy of answers and their relevance to business needs.

[0116] Multiple candidate answers are structured and organized using a chronological numbering mechanism to ensure the orderliness and traceability of the output, facilitating users' systematic viewing and comparison of different generated content. Simultaneously, annotation information is added to each answer before it is returned to the user. This step, by introducing a source annotation mechanism, clearly identifies the generation attributes of the answers, effectively manages user expectations, and enhances the transparency and credibility of the system output, avoiding misunderstandings or misuse of automatically generated content. In practice, the generated answers undergo systematic screening and post-processing to ensure they meet user needs and expectations. When the large model generates multiple candidate answers, the system strictly sorts them according to the chronological order of their generation and adds a corresponding number to each answer to form an ordered list returned to the user. Each answer includes clear standard prompts, ensuring users clearly understand the source and reference nature of the answer, ultimately achieving a user experience that provides rich information while clearly defining responsibility boundaries.

[0117] Based on the same inventive concept, embodiments of this specification also provide a business question-answering device based on a large model, such as... Figure 6 As shown, it includes:

[0118] FAQ question-and-answer engine module 601 is used for preliminary question-and-answer processing based on the FAQ question-and-answer engine;

[0119] The vectorization unit 602 is used to vectorize the user question through a large language model and convert it into a vectorized question with a high-dimensional vector representation when the matching accuracy between the FAQ question-answering engine's answer and the user question is lower than a first threshold.

[0120] The retrieval unit 603 is used to retrieve the document paragraphs most relevant to the vectorization problem from pre-vectorized document data, the document data including vectorized segments of the system manual and vectorized segments of the knowledge base;

[0121] Answer generation unit 604 is used to dynamically generate an answer that matches the user's question based on at least one retrieved document paragraph using a large language model;

[0122] Answer return unit 605 is used to generate a final answer based on the answer that matches the user's question and return it to the user.

[0123] Since the problem-solving principle of the business question-answering device based on the large model is similar to that of the business question-answering method based on the large model, the implementation of this device can be found in the above-mentioned instruction manual, and repeated parts will not be described again.

[0124] In one embodiment of this specification, a computer device is also provided for implementing the methods described in any of the above embodiments, such as... Figure 7The diagram illustrates the structure of a computer device according to an embodiment of this specification. The computer device 702 may include one or more processors 704, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 702 may also include any memory 706 for storing information of any kind, such as code, settings, data, etc. Without limitation, for example, the memory 706 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of the computer device 702. In one case, when the processor 704 executes associated instructions stored in any memory or combination of memories, the computer device 702 may perform any operation of the associated instructions. The computer device 702 also includes one or more drive mechanisms 708 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.

[0125] Computer device 702 may also include an input / output module 710 (I / O) for receiving various inputs (via input device 712) and providing various outputs (via output device 714). A specific output mechanism may include a presentation device 716 and an associated graphical user interface (GUI) 718. In other embodiments, the input / output module 710 (I / O), input device 712, and output device 714 may be omitted, and the device may function solely as a computer device within a network. Computer device 702 may also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the components described above together.

[0126] Communication link 722 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 722 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0127] Corresponding to Figures 2 to 5 In addition to the methods described above, embodiments of this specification also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the methods described above.

[0128] This specification also provides computer-readable instructions, wherein when a processor executes the instructions, the program therein causes the processor to perform the following...Figures 2 to 5 The method shown.

[0129] It should be understood that in the various embodiments of this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0130] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this specification generally indicates that the preceding and following related objects have an "or" relationship.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments in this specification.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described in this specification, depending on actual needs.

[0135] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0136] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this specification, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] This specification uses specific embodiments to illustrate the principles and implementation methods of this specification. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of the embodiments in this specification. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments in this specification. Therefore, the content of this specification should not be construed as a limitation on the embodiments in this specification.

Claims

1. A large model-based business question answering method, characterized by, After the preliminary question and answer processing based on the FAQ question and answer engine, further comprising, When the matching accuracy of the answer of the FAQ question and answer engine with the user question is lower than a first threshold, the user question is vectorized by a large language model into a vectorized question represented by a high-dimensional vector; Retrieving the most relevant document paragraph from the pre-vectorized document data, wherein the document data contains vectorized segments of system manuals and vectorized segments of knowledge bases; Based on the retrieved at least one document paragraph, dynamically generating an answer conforming to the user question by using a large language model; Generating a final answer according to the answer conforming to the user question and returning it to the user.

2. The large model-based business Q&A method of claim 1, wherein, The preliminary question and answer processing based on the FAQ question and answer engine further comprises, Retrieving the most similar at least one similar question from the corpus based on the keyword matching recall algorithm; De-duplicating and mapping the similar question, and retrieving a pre-set answer matching the similar question. 3.The large model-based business Q&A method of claim 1, wherein, Retrieving the most relevant document paragraph from the pre-vectorized document data further comprises, Segmenting the contents of the system manual and the knowledge base, and assigning a unique identifier to each segment of content; Converting each segment of content carrying the identifier into a vectorized document paragraph represented by a high-dimensional vector.

4. The large model-based business Q&A method of claim 3, wherein, Retrieving the most relevant document paragraph from the pre-vectorized document data further comprises, Storing the vectorized document paragraph in a vector database; Based on the approximate nearest neighbor search algorithm, constructing an efficient vector retrieval index for all data in the vector database. 5.The large model-based business Q&A method according to claim 1, wherein, Vectorizing the user question by a large language model into a vectorized question represented by a high-dimensional vector further comprises, Using a pre-trained language model based on the Transformer architecture to map the text input of the user question into a high-dimensional vector representation containing semantic information through semantic encoding capability.

6. The large model-based business Q&A method of claim 5, wherein, Dynamically generating an answer conforming to the user question by using a large language model further comprises, Retrieving whether the most relevant document paragraph has context information; If it has context information, constructing the retrieval result and the context information into an answer conforming to the user question through natural language generation technology; If it does not have context information, using a large language model based on the Transformer architecture to construct the document paragraph into an answer conforming to the user question.

7. The large model-based business Q&A method according to claim 6, characterized in that, If it does not have context information, further comprising, Calculating the similarity score between the vectorized question and the vectorized paragraph in the document database; Based on the similarity score, filtering out one or more document paragraphs with the highest relevance as target data; Judging whether the highest similarity score of the target data is higher than a second pre-set threshold; If not, determining that the retrieval fails, generating and returning a retrieval failure prompt information; If yes, inputting the target data into a large language model to generate the answer conforming to the user question. 8.The large model-based business Q&A method according to claim 7, wherein, Inputting the target data into a large language model to generate the answer conforming to the user question further comprises, retrieve, in the target data, top N pieces of candidate information most similar to the vectorized question based on vector similarity, N being a set positive integer; add the top N pieces of candidate information and the user question to a first prompt instruction prompt, and submit to a general large model to instruct the general large model to generate an answer most matched to the user question, and then compose a final answer with the user question. 9.A large model-based business question answering apparatus, characterized by comprising: The device comprises: an FAQ question and answer engine module configured to perform preliminary question and answer processing based on an FAQ question and answer engine; a vectorization unit configured to, when the matching accuracy of the answer of the FAQ question and answer engine to the user question is lower than a first threshold, vectorize the user question through a large language model to convert the user question into a vectorized question represented by a high-dimensional vector; a retrieval unit configured to retrieve, in pre-vectorized document data, a document paragraph most relevant to the vectorized question, the document data including vectorized segments of a system manual and vectorized segments of a knowledge base; an answer generation unit configured to dynamically generate an answer conforming to the user question based on the retrieved at least one document paragraph by using a large language model; an answer returning unit configured to generate a final answer according to the answer conforming to the user question and return the final answer to the user.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor of the computer device to implement the method of any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor of the computer device to implement the method of any one of claims 1 to 8.

Images