Automatic question answering method and device, electronic equipment and storage medium

Through the automatic question-and-answer method in the insurance and finance field, combined with knowledge base matching, intention recognition and inference models, the problem that traditional technology is difficult to effectively automatic question-and-answer is solved, and intelligent question-and-answer with high accuracy and high coverage is achieved, which is suitable for vertical fields. Requirements, save resources.

CN119088931APending Publication Date: 2024-12-06CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411194255.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the field of insurance and finance, it is difficult for traditional technologies to effectively conduct automatic Q&A, especially in underwriting scenarios with high professionalism. There are bottlenecks in the application of existing large models in vertical fields, which cannot effectively cover insurance underwriting knowledge, professionalism and accuracy of content. Insufficient sex.

Method used

Provide an automatic question-and-answer method, by obtaining user questions, first match in the preset knowledge base, and generate answers when the match is successful; when the match is unsuccessful, input the question into the pre-trained intent distribution model to generate intent recognition results. And build prompt words based on knowledge tools and determine the answer through inference model. This method combines knowledge base matching, intention recognition, prompt word construction and reasoning modules to adapt to the needs of vertical fields to achieve intelligent Q&A.

Benefits of technology

It has achieved 100% coverage of Tuancai products, 50% manual replacement rate, and improved the answer accuracy. Compared with traditional RAG question and answer accuracy, it has saved resources and is suitable for tables, maps, pictures and other knowledge types. The architecture is modular and adjustable, and the inference module does not require training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088931A_ABST
    Figure CN119088931A_ABST
Patent Text Reader

Abstract

The invention discloses an automatic question answering method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a user question, carrying out the matching of the user question based on a preset knowledge base, and generating an answer when the matching succeeds; when the matching is not successful, the user question is input into a pre-trained complete intention distribution model to generate an intention recognition result, the intention distribution model is matched with a scene corresponding to the user question, and the intention recognition result is a knowledge tool or an answer corresponding to the user question; on the basis of the knowledge tool, generating a retrieval result corresponding to the user question, and constructing a first prompt word on the basis of the retrieval result; and based on the first cue word, determining an answer through a pre-configured reasoning model. The technical problem that automatic question answering cannot be well performed in the underwriting field in the prior art is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an automatic question-answering method, device, electronic equipment and storage medium. Background Art

[0002] In the insurance and finance field, there are many types of insurance, reaching more than 6,000, and they are highly professional and difficult for the team to learn. This puts a lot of pressure on the underwriter service team, and there is an urgent need for automation technology to assist underwriters in reducing their burden.

[0003] Traditional technical means are difficult to solve complex scenarios such as underwriting scenarios that require high professionalism and logical reasoning, while traditional large model technology faces the problem of generating assumptions and questionable authenticity. At present, the application of large models in the vertical domain has always been a bottleneck.

[0004] Fine-tuning large language models in vertical fields has extremely high requirements for the amount, professionalism, and coverage of fine-tuning data. Existing open source datasets such as BELLA and Guanaco only contain general knowledge, and do not cover vertical fields such as insurance underwriting knowledge enough. In addition, the content is not professional and accurate enough, so customized fine-tuning in vertical fields is required. However, the order of magnitude of fine-tuning data is very large, and the training cost is high, time-consuming, and costly. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide an automatic question-answering method, device, electronic device and storage medium to solve the technical problem that the prior art cannot perform automatic question-answering well in the underwriting field.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides an automatic question answering method, comprising the following steps:

[0008] Obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful;

[0009] When the match is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question;

[0010] Based on the knowledge tool, generating a search result corresponding to the user question, and constructing a first prompt word based on the search result;

[0011] Based on the first prompt word, an answer is determined by a pre-configured reasoning model.

[0012] In some embodiments, the obtaining of the user question, matching the user question based on a preset knowledge base, and generating an answer when the match is successful, includes:

[0013] Obtaining user questions, and inputting the user questions into the question-answering system of the knowledge base;

[0014] Using a preset matching model to match the user question to obtain a matching result;

[0015] Based on the matching result, it is determined whether the match is successful, and when the match is successful, an answer is generated.

[0016] In some embodiments, judging whether the match is successful based on the matching result includes:

[0017] A similarity threshold is obtained, and the matching result is compared with the similarity threshold. When the matching result is not less than the similarity threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails.

[0018] In some embodiments, the intention distribution model is established based on thought chain data, the input of the intention distribution model is user questions, and the output of the intention distribution model is a knowledge tool.

[0019] In some embodiments, when the matching is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, including:

[0020] When the match is unsuccessful, generating a second prompt word based on the user question;

[0021] Determining whether an answer can be generated according to the second prompt word through the intention distribution model, and if so, directly generating the answer through the intention distribution model;

[0022] When the answer cannot be generated directly, the knowledge tool name and the corresponding knowledge tool calling command are output through the intention distribution model.

[0023] In some embodiments, generating a search result corresponding to the user question based on the knowledge tool, and constructing a first prompt word based on the search result includes:

[0024] Based on the intention recognition result, retrieve the knowledge tool corresponding to the intention recognition result to search the user question to obtain a search result;

[0025] The search result is processed to obtain a first prompt word.

[0026] In some embodiments, the intent distribution model is a ChatGLM-6B model, and the reasoning model is a Qwen-72B model.

[0027] In a second aspect, the present invention further provides an automatic question-answering device, comprising:

[0028] The knowledge base matching module is used to obtain user questions, match the user questions based on the preset knowledge base, and generate answers when the match is successful;

[0029] An intent recognition module, which is used to input the user question into a pre-trained intent distribution model to generate an intent recognition result when the match is unsuccessful, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question;

[0030] A prompt word construction module, used to generate a search result corresponding to the user question based on the knowledge tool, and construct a first prompt word based on the search result;

[0031] The reasoning module is used to determine the answer based on the first prompt word through a pre-configured reasoning model.

[0032] In a third aspect, the present invention further provides an electronic device, comprising: a processor and a memory;

[0033] The memory stores a computer program executable by the processor;

[0034] When the processor executes the computer program, the steps in the automatic question-answering method described above are implemented.

[0035] In a fourth aspect, the present invention further provides a computer-readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the steps in the automatic question-answering method as described above.

[0036] Compared with the prior art, the automatic question-answering method, device, electronic device and storage medium provided by the present invention first obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful; then when the match is unsuccessful, the user questions are input into a pre-trained intent distribution model to generate intent recognition results, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question; then, based on the knowledge tool, a retrieval result corresponding to the user question is generated, and a first prompt word is constructed based on the retrieval result; finally, based on the first prompt word, the answer is determined through a pre-configured reasoning model. It can achieve 100% coverage of group finance products, 50% manual replacement rate, realize intelligent question answering, and improve the accuracy of answers. Compared with traditional RAG question and answer, the accuracy is increased by more than 20%. In addition, it improves the efficiency of knowledge use and is compatible with tables, graphs, pictures, QA and other knowledge types. In addition, all architectures of the present invention are spliced ​​in the form of modules. For new needs or problems, only adjustments need to be made to each sub-module. At the same time, the heaviest reasoning module does not require training and fine-tuning, saving a lot of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of an automatic question-answering method provided by an embodiment of the present invention;

[0038] Figure 2 is a flowchart of step S100 in the automatic question-answering method provided by an embodiment of the present invention;

[0039] Figure 3 is a flowchart of step S200 in the automatic question-answering method provided by an embodiment of the present invention;

[0040] Figure 4 is a flowchart of step S300 in the automatic question-answering method provided by an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of functional modules of an automatic question-answering device provided by an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] In the insurance and finance sector, there are many types of insurance, reaching more than 6,000, and they are highly professional and difficult for the team to learn. Therefore, the underwriter service team is under great pressure, and there is an urgent need for automated technology to assist underwriters in reducing their burden. Traditional technical means are difficult to solve complex scenarios such as underwriting scenarios that are highly professional and require logical reasoning, while traditional large model technology faces the problem of generating assumptions and questionable authenticity. At present, the application of large models in vertical fields has always been a bottleneck, resulting in insufficient professionalism and accuracy of current model content.

[0045] See also Figure 1 , Figure 1 The figure is a flowchart of an automatic question-and-answer method provided by an embodiment of the present invention. The automatic question-and-answer method involved in the present invention can be used in automatic question-and-answer in the field of underwriting to perform automatic question-and-answer in a professional and accurate manner. The automatic question-and-answer method involved in the present invention can be executed by an electronic device that can perform operations such as receiving or sending data, and can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, portable computers, desktop servers, etc. Figure 1 As shown, the method specifically includes the following steps S100 to S400.

[0046] S100, obtaining a user question, matching the user question based on a preset knowledge base, and generating an answer when the match is successful.

[0047] In this embodiment, the user first initiates a question through the client device, that is, inputs the user question. After obtaining the user question, the system first responds to the user question based on a preset knowledge base. If it can be directly matched, there is no need to go through the subsequent process, and the answer required by the user can be directly generated. If it cannot be obtained, it is necessary to execute step S200.

[0048] The knowledge base is manually written from insurance-related documents, or constructed by extracting common questions from communications between sales staff and underwriters. In other words, the knowledge base is established based on user experience and can respond quickly to user questions.

[0049] S200. When the match is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question.

[0050] In this embodiment, when the match is unsuccessful, it is necessary to obtain the answer in the way of thought chain. Specifically, the traditional RAG method is constructed by slicing the document and putting it into the model as a prompt. The problem with this approach is that not all knowledge is stored in the form of documents. Some is only extracted in the form of tables, QA, graphs, etc., and this fine-grained knowledge is often more suitable for model understanding and has a higher degree of match with the problem. In order to make better use of this part of knowledge, the present invention combines the idea of ​​COT (thinking chain) and designs a modular RAG question-answering architecture to improve the accuracy and iterability of question-answering.

[0051] S300: Generate a search result corresponding to the user question based on the knowledge tool, and construct a first prompt word based on the search result.

[0052] In this embodiment, the search results are generated accordingly through the knowledge tool, and then the first prompt word is generated according to the result output by the tool. The subsequent reasoning model can determine the answer according to the first prompt word.

[0053] S400: Based on the first prompt word, determine an answer through a pre-configured reasoning model.

[0054] In this embodiment, after the first prompt word is determined, the answer can be determined based on a pre-established reasoning model. The reasoning model does not need to be trained, and different base models can be switched at will as reasoning models. Among them, the knowledge tool is selected according to different user questions through the intent distribution model, and then queried according to the user question, and then the relevant content is returned. Therefore, the input of the reasoning model is the user question and the retrieval results of the knowledge tool, and the output is the answer to the question.

[0055] In the embodiment of the present invention, the user question is first obtained, and the user question is matched based on the preset knowledge base. When the match is successful, an answer is generated; then when the match is unsuccessful, the user question is input into the pre-trained intention distribution model to generate an intention recognition result, wherein the intention distribution model is adapted to the scene corresponding to the user question, and the intention recognition result is a knowledge tool or answer corresponding to the user question; then based on the knowledge tool, a search result corresponding to the user question is generated, and a first prompt word is constructed based on the search result; finally, based on the first prompt word, the answer is determined by a pre-configured reasoning model. It can achieve 100% coverage of group financial products, 50% manual replacement rate, realize intelligent question answering, and improve the accuracy of answering, which is more than 20% higher than the traditional RAG question answering accuracy. In addition, the efficiency of knowledge use is improved, and it can also be compatible with knowledge types such as tables, maps, pictures, QA, etc. In addition, all architectures of the present invention are spliced ​​in the form of modules, and only adjustments need to be made in each submodule for new needs or problems. At the same time, the heaviest reasoning module does not need to be trained and fine-tuned, saving a lot of resources.

[0056] In some embodiments, see Figure 2 , the step S100 specifically includes:

[0057] S110, obtaining a user question, and inputting the user question into the question-answering system of the knowledge base;

[0058] S120, using a preset matching model to match the user question to obtain a matching result;

[0059] S130. Based on the matching result, determine whether the match is successful, and if the match is successful, generate an answer.

[0060] In this embodiment, the user's questions are matched through the question-answering system (QA) of the knowledge base to quickly obtain the answers required by the user. Specifically, a matching model is preset in the system, and the matching result can be calculated through the matching model, and then it can be determined whether the matching is successful based on the matching result.

[0061] Optionally, the matching model is established based on the cosine similarity of word2vec, which is a method for calculating the similarity between texts through word vectors. This method uses the Word2Vec model to convert words into vector form, and then measures the semantic similarity of the text by calculating the cosine value between the vectors. This is widely used in natural language processing (NLP), such as text clustering, recommendation systems, etc.

[0062] Word2Vec is a popular word embedding technology that converts text into dense vectors so that semantically similar words are closer in the vector space. There are two main architectures for this model: Continuous Bag of Words (CBOW) and Skip-gram. CBOW predicts the current word through the context, while Skip-gram predicts the context through the current word.

[0063] In order to calculate cosine similarity using Word2Vec, the text needs to be preprocessed. This includes steps such as word segmentation and removal of stop words to ensure data quality. For example, the Chinese text can be segmented using the jieba word segmentation tool. After word segmentation, each word is mapped to a fixed vector that can be used for further mathematical operations.

[0064] After obtaining the vector representation of the word, the common practice is to average the vectors of all the words in a sentence as the vector representation of the sentence. In this way, each sentence is converted into a vector, and then the cosine similarity between the two vectors can be calculated. The formula for calculating cosine similarity is Where A and B are two vectors, "·" represents the dot product, and "||" represents the modulus of the vector.

[0065] In addition, there are some improved methods to improve the effect. For example, Paragraph2Vec (Doc2Vec) is an extended method that can directly process longer texts and solve the problem of phrase or sentence comparison. Doc2Vec uses the same idea as Word2Vec, but considers not only the vector of the word, but also the vector representation of the entire document, which is more effective for calculating the similarity of long texts.

[0066] Therefore, cosine similarity calculation based on Word2Vec is a powerful natural language processing method that can effectively capture the semantic relationship between texts. Through appropriate text preprocessing and parameter adjustment, the accuracy of calculation and the breadth of application can be significantly improved.

[0067] In some embodiments, judging whether the match is successful based on the matching result includes:

[0068] A similarity threshold is obtained, and the matching result is compared with the similarity threshold. When the matching result is not less than the similarity threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails.

[0069] In this embodiment, whether the match is successful is determined by the cosine similarity based on word2vec. For example, when the similarity result exceeds a threshold value of 0.98, the match is determined to be successful, and the answer can be directly output at this time. Otherwise, the match is determined to have failed.

[0070] In some embodiments, in step S200, the intention distribution model is established based on chain of thought (COT) data, the input of the intention distribution model is user questions, and the output of the intention distribution model is a knowledge tool.

[0071] In this embodiment, COT data is constructed according to different scenarios, and the small model (ChatGLM-6B) is trained to learn to select tools. Each tool is adapted to different scenarios or data types, and the small model (intention distribution model) performs distribution work, selects one or more corresponding tools for different problems, and generates corresponding tool commands.

[0072] Specifically, COT data refers to a dataset or method that uses the Chain-of-Thought (CoT) technique to guide a large language model (LLM) to perform complex reasoning tasks. This method enhances the model's arithmetic, common sense, and symbolic reasoning capabilities by explicitly outputting step-by-step reasoning steps before answering questions.

[0073] The CoT technology was proposed to address the shortcomings of large-scale language models in dealing with complex tasks that require multi-step reasoning. Traditional prompt learning usually only focuses on the direct mapping from input to output, while CoT builds a process from input to thinking chain to output by adding intermediate reasoning steps. This improvement significantly improves the performance of the model in various complex reasoning tasks. Its core is to imitate the thinking process of humans in solving problems, that is, to get the final answer through a series of thinking, analysis and reasoning steps. This approach not only improves the accuracy of the model, but also enhances the interpretability of decisions.

[0074] CoT technology is divided into two methods: Zero-shot CoT and Few-shot CoT. Zero-shot CoT does not require manually labeled examples, but allows the model to automatically generate reasoning steps and answers through specific prompts such as "Let's think step by step". In contrast, Few-shot CoT requires some labeled examples to guide the model to perform similar reasoning. Both methods have their own advantages, but Few-shot CoT usually brings better performance improvements.

[0075] In order to further improve the effect of CoT technology, the Self-Consistency method can be used to generate multiple reasoning paths and answers through multiple decoding samples, and then use a voting strategy to select the best answer. In addition, there is also a method for automatically building CoT (Auto-CoT), which uses simplified prompts to prompt the model to generate reasoning paths by itself, thereby reducing labor costs.

[0076] Therefore, the embodiment of the present invention uses COT data to train the intent distribution model, and the CoT technology significantly improves the performance of large language models in complex reasoning tasks by introducing intermediate reasoning steps. This not only improves the accuracy of the model, but also enhances the interpretability of the results.

[0077] For example, the embodiment of the present invention organizes historical underwriting-related consulting questions into documents, inputs them into the GPT model, and summarizes them into 22 categories, such as product highlights, product comparisons, hesitation periods, and other scenarios. Then, COT templates are constructed according to different scenarios. In order to make the subsequent training model responses more accurate, the present invention utilizes the Chain Of Thought capability of the GPT model to allow the model to show its decision-making process. At the same time, it develops tools such as insurance knowledge retrieval, product recommendations, and industry and job risk inquiries, allowing the model to select tools and use external knowledge to assist in decision-making. Specifically, first, the product highlight COT data is generated after fine-tuning the prompt of the intent distribution model. Then, GPT is used to generate consulting sentences for each scenario, and the special information slots are marked. GPT is then used to generalize the content of different information slots to improve the adaptability of information to real scenarios. Taking the product slot as an example, the standard is the standard name of the product, and the entity is the general expression of the product generalized by GPT. Finally, the COT template and the slot information of STEP3-4 are randomly matched to automatically generate COT training data. Therefore, the input of the small model is Human, and the output is Inner thought and command, where "Human" is the user's question, "Inner Thoughts" is the model decision-making process, "Commands" is the tool selected by the model, and "Node Name" is the scenario type.

[0078] In some embodiments, the intent distribution model is a ChatGLM-6B model, which is a conversational language model that supports Chinese and English bilingual question and answer, has 6.2 billion parameters, and can be locally deployed on consumer-grade graphics cards. Specifically, ChatGLM-6B is an open source bilingual conversational language model based on the Genera l Language Model l (GLM) architecture, which uses the same technology as ChatGLM and is optimized for Chinese question and answer and conversation.

[0079] ChatGLM-6B combines model quantization technology, allowing users to deploy it locally on consumer-grade graphics cards (with a minimum of 6GB of video memory at the INT4 quantization level). This quantization technology allows the model to maintain efficient computing while reducing the demand for hardware resources.

[0080] During training, ChatGLM-6B can generate answers that are quite in line with human preferences after about 1T of Chinese and English bilingual training, supplemented by supervised fine-tuning, feedback self-help, human feedback reinforcement learning and other technologies. These training methods ensure the model's advantages in understanding and generating natural language.

[0081] In addition, ChatGLM-6B supports bilingual question and answer in Chinese and English, which makes its application more extensive and flexible in different language environments. The model also provides web version demo and command line demo, which facilitates users to have an interactive conversation experience. For example, you can start the web version demo based on Grad io by running python web_demo.py. In addition, users can not only download the model through Hugging Face Hub, but also load the model locally, which greatly improves the accessibility and flexibility of the model.

[0082] Furthermore, ChatGLM-6B can be used for various text generation tasks, such as automatic writing, content creation, etc. Due to its excellent question-answering and conversation capabilities, the model is suitable for building intelligent dialogue systems, such as customer service robots, virtual assistants, etc. Moreover, the bilingual support of Chinese and English enables ChatGLM-6B to be applied to cross-language information conversion and translation tasks.

[0083] In some embodiments, the knowledge tools refer to software or platforms that can help individuals or organizations acquire, organize, store and share knowledge.

[0084] Furthermore, the output of the intent recognition model also includes the model decision process. For details, please refer to Figure 3 , the step S200 specifically includes:

[0085] S210, when the match is unsuccessful, generating a second prompt word based on the user question;

[0086] S220: Determine whether an answer can be generated according to the second prompt word through the intention distribution model; if so, directly generate the answer through the intention distribution model;

[0087] S230: When the answer cannot be generated directly, the knowledge tool name and the corresponding knowledge tool calling command are output through the intention distribution model.

[0088] In this embodiment, the intent distribution model can directly output the decision-making process, that is, directly output the answer, or use external tools to assist in decision-making. Specifically, the user question slice is first put into the prompt word, and then the prompt word is input into the intent distribution model. The intent distribution model then makes a decision to determine whether it can be replied directly. If the intent distribution model can make a decision directly at this time, it will directly reply with the answer. If it cannot make a decision directly, it is necessary to use external knowledge tools to assist in decision-making. The corresponding knowledge tool name is output, that is, the user question selects one or more corresponding tools, and generates corresponding tool commands.

[0089] In some embodiments, see Figure 3 , the step S300 specifically includes:

[0090] S310, based on the intention recognition result, retrieve the knowledge tool corresponding to the intention recognition result to search the user question to obtain a search result;

[0091] S320: Process the search result to obtain a first prompt word.

[0092] In this embodiment, after obtaining the name of the knowledge tool, the knowledge tool can be called through the knowledge tool command to search for the user question. The corresponding search results can be obtained through the knowledge tool, and then similar to the traditional RAG method, the search results are sliced ​​and processed to obtain the first prompt word. The answer can be obtained based on the first prompt word, and the first prompt word obtained at this time is more relevant to the user question and can better reflect the answer the user wants to obtain.

[0093] Furthermore, the prompting technology described in the present invention is a new method that combines the question-answering training data set, the underwriting professional knowledge base, and the user's personal information, which can greatly improve the quality of the inference model's answer. At the same time, the semantic similarity query technology is used to generate text embeddings for the question itself and the information derived from the question (Self-generation chain of thought) (actual operations such as "underwriting-related content that may be involved in answering this question"), and by quickly retrieving similar content, a high-quality prompting word template of multi-angle information is dynamically constructed to enhance the accuracy of the large model in answering questions and reduce the possibility of "hallucination" (hallucination) of the large model.

[0094] In some embodiments, in step S400, the inference model is a universal large model, which does not need to be trained, and different base models can be switched at will as the inference model. Optionally, the inference model is a Qwen-72B model.

[0095] In this embodiment, Qwen-72B is a 72 billion parameter model in the Tongyi Qianwen large model series. Qwen-72B is based on the Transformer architecture and is trained with ultra-large-scale pre-training data, covering high-quality Chinese, English, multilingual, code, and mathematical data types. It has strong performance and extensive vocabulary support, and can surpass existing open source models on multiple downstream evaluation tasks. The following will introduce various aspects of the model in detail:

[0096] 1. Technical features

[0097] Infrastructure: Qwen-72B is a large language model based on Transformer, which has significant advantages in processing natural language tasks.

[0098] Training data: Pre-training is done using data from more than 3 trillion tokens, including online texts, professional books, codes, etc., covering both general and professional fields.

[0099] Context length: Supports a context length of 32k, which enables the model to handle more complex long text tasks.

[0100] Performance optimization: In multiple downstream evaluation tasks, such as common sense reasoning, code generation, and mathematical problem solving, the results are significantly better than other open source models.

[0101] 2. Application fields

[0102] Language understanding and generation: Qwen-72B performs well in language understanding and generation benchmarks and can be widely used in scenarios such as chatbots and content creation.

[0103] Coding capability: After being trained with a large amount of code data, Qwen-72B has powerful code generation capabilities, which is suitable for tasks such as automatic programming and code error repair.

[0104] Multi-language support: It uses a vocabulary of about 150,000 words, which is friendly to multiple languages ​​and facilitates users to develop cross-language applications.

[0105] Mathematical ability: They also demonstrate excellent ability in solving mathematical problems, which can be used to solve problems in education, scientific research and other fields.

[0106] 3. Technical Support

[0107] Distributed training: supports technologies such as data parallelism, operator splitting, pipeline parallelism, and sequence parallelism, greatly improving the training efficiency of large models.

[0108] Optimization technology: Advanced technologies such as selective activation recalculation and ZeRO memory optimization are used to ensure efficient training process and lower resource consumption.

[0109] Fine-tuning tools: Provide convenient fine-tuning tools, users can fine-tune the model based on specific tasks to adapt to different business needs.

[0110] Deployment application: Supports one-click deployment, and users can quickly apply trained models to actual business scenarios.

[0111] 4. Platform Services

[0112] API service: Through the API service provided by the DashScope platform, developers can easily call Qwen-72B to perform various inference tasks.

[0113] Huggi ngFace support: A Qwen-72B model library is provided on the Huggi ngFace platform to facilitate developers to load and use models.

[0114] Performance optimization tools: It is recommended to install the flash-attention library to improve the inference efficiency of the model and reduce the memory usage.

[0115] vLLM support: Using vLLM enables inference with longer context lengths and higher generation speedup.

[0116] 5. User Experience

[0117] Multi-round dialogue interaction: Qwen-72B-Chat supports multi-round dialogue interaction, and can adjust role-playing, language style transfer and other functions through system commands.

[0118] Environment configuration: Python 3.8 and above and PyTorch 1.12 and above are required. CUDA 11.4 and above are recommended.

[0119] Example guidance: The official provides detailed usage examples and quick start guides to help users get started quickly.

[0120] In summary, Qwen-72B, with its huge parameter scale and advanced technical architecture, has demonstrated excellent performance and application potential in multiple fields, and is particularly suitable for scenarios with complex language processing requirements.

[0121] The technical solution provided by the present invention first obtains the user question, matches the user question based on the preset knowledge base, and generates an answer when the match is successful; then when the match is unsuccessful, the user question is input into the pre-trained intention distribution model to generate an intention recognition result, wherein the intention distribution model is adapted to the scene corresponding to the user question, and the intention recognition result is the knowledge tool or answer corresponding to the user question; then based on the knowledge tool, the search result corresponding to the user question is generated, and the first prompt word is constructed based on the search result; finally, based on the first prompt word, the answer is determined by the pre-configured reasoning model. It can achieve 100% coverage of group financial products, 50% manual replacement rate, realize intelligent question answering, and improve the accuracy of answering, which is more than 20% higher than the traditional RAG question answering accuracy. In addition, the efficiency of knowledge use is improved, and it can also be compatible with knowledge types such as tables, maps, pictures, QA, etc. In addition, all architectures of the present invention are spliced ​​in the form of modules, and only adjustments need to be made in each submodule for new needs or problems. At the same time, the heaviest reasoning module does not need to be trained and fine-tuned, saving a lot of resources.

[0122] Another embodiment of the present invention provides an automatic question-answering device, see Figure 5 The automatic question-answering device includes a knowledge base matching module 11, an intention recognition module 12, a prompt word construction module 13 and a reasoning module 14.

[0123] The knowledge base matching module 11 is used to obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful.

[0124] The intent recognition module 12 is used to input the user question into a pre-trained intent distribution model to generate an intent recognition result when the match is unsuccessful, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question.

[0125] The prompt word construction module 13 is used to generate a search result corresponding to the user question based on the knowledge tool, and to construct a first prompt word based on the search result.

[0126] The reasoning module 14 is used to determine the answer based on the first prompt word through a pre-configured reasoning model.

[0127] In this embodiment, firstly, the user question is obtained, and the user question is matched based on the preset knowledge base. When the match is successful, an answer is generated; then when the match is unsuccessful, the user question is input into the pre-trained intention distribution model to generate an intention recognition result, wherein the intention distribution model is adapted to the scene corresponding to the user question, and the intention recognition result is the knowledge tool or answer corresponding to the user question; then based on the knowledge tool, the search result corresponding to the user question is generated, and the first prompt word is constructed based on the search result; finally, based on the first prompt word, the answer is determined by the pre-configured reasoning model. It can achieve 100% coverage of group financial products, 50% manual replacement rate, realize intelligent question answering, and improve the accuracy of answering, which is more than 20% higher than the traditional RAG question answering accuracy. In addition, the efficiency of knowledge use is improved, and it can also be compatible with knowledge types such as tables, maps, pictures, QA, etc. In addition, all the architectures of the present invention are spliced ​​in the form of modules, and only adjustments need to be made in each submodule for new needs or problems. At the same time, the heaviest reasoning module does not need to be trained and fine-tuned, saving a lot of resources.

[0128] It should be noted that the module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions, which are more suitable for describing the execution process of automatic question answering than programs. For the specific implementation methods of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0129] In some embodiments, the knowledge base matching module 11 is specifically used for:

[0130] Obtaining user questions, and inputting the user questions into the question-answering system of the knowledge base;

[0131] Using a preset matching model to match the user question to obtain a matching result;

[0132] Based on the matching result, it is determined whether the match is successful, and when the match is successful, an answer is generated.

[0133] In some embodiments, judging whether the match is successful based on the matching result includes:

[0134] A similarity threshold is obtained, and the matching result is compared with the similarity threshold. When the matching result is not less than the similarity threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails.

[0135] In some embodiments, the intention distribution model is established based on thought chain data, the input of the intention distribution model is user questions, and the output of the intention distribution model is a knowledge tool.

[0136] In some embodiments, the intention recognition module 12 is specifically used to:

[0137] When the match is unsuccessful, generating a second prompt word based on the user question;

[0138] Determining whether an answer can be generated according to the second prompt word through the intention distribution model, and if so, directly generating the answer through the intention distribution model;

[0139] When the answer cannot be generated directly, the knowledge tool name and the corresponding knowledge tool calling command are output through the intention distribution model.

[0140] In some embodiments, the prompt word construction module 13 is specifically used for:

[0141] Based on the intention recognition result, retrieve the knowledge tool corresponding to the intention recognition result to search the user question to obtain a search result;

[0142] The search result is processed to obtain a first prompt word.

[0143] In some embodiments, the intent distribution model is a ChatGLM-6B model, and the reasoning model is a Qwen-72B model.

[0144] Another embodiment of the present invention provides an electronic device, such as Figure 6 As shown, the electronic device 10 includes:

[0145] One or more processors 110 and memory 120, Figure 6 A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0146] The processor 110 is used to complete various control logics of the electronic device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0147] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the automatic question-answering method in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the electronic device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, the automatic question-answering method in the above method embodiment is implemented.

[0148] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating platform, an application required for at least one function; the data storage area may store data created according to the use of the electronic device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the electronic device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] One or more units are stored in the memory 120, and when executed by one or more processors 110, the automatic question answering method in any of the above method embodiments is executed, for example, the automatic question answering method described above is executed. Figure 1 Method steps in

[0150] Obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful;

[0151] When the match is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question;

[0152] Based on the knowledge tool, generating a search result corresponding to the user question, and constructing a first prompt word based on the search result;

[0153] Based on the first prompt word, an answer is determined by a preconfigured reasoning model.

[0154] In some embodiments, the processor 110 further performs the following steps:

[0155] Obtaining user questions, and inputting the user questions into the question-answering system of the knowledge base;

[0156] Using a preset matching model to match the user question to obtain a matching result;

[0157] Based on the matching result, it is determined whether the match is successful, and when the match is successful, an answer is generated.

[0158] In some embodiments, the processor 110 further performs the following steps:

[0159] A similarity threshold is obtained, and the matching result is compared with the similarity threshold. When the matching result is not less than the similarity threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails.

[0160] In some embodiments, the processor 110 further performs the following steps:

[0161] When the match is unsuccessful, generating a second prompt word based on the user question;

[0162] Determining whether an answer can be generated according to the second prompt word through the intention distribution model, and if so, directly generating the answer through the intention distribution model;

[0163] When the answer cannot be generated directly, the knowledge tool name and the corresponding knowledge tool calling command are output through the intention distribution model.

[0164] In some embodiments, the processor 110 further performs the following steps:

[0165] Based on the intention recognition result, retrieve the knowledge tool corresponding to the intention recognition result to search the user question to obtain a search result;

[0166] The search result is processed to obtain a first prompt word.

[0167] Those skilled in the art will understand that Figure 6 The embodiments of the electronic device shown in the figure do not constitute a limitation on the specific composition of the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the electronic device may include a memory, a processor, and a network interface. In such embodiments, the structure and function of the memory and the processor are similar to those in the figure. Figure 6 The embodiments shown are consistent and will not be described again here.

[0168] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the automatic question-answering method disclosed in the embodiment of the present invention is implemented.

[0169] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0170] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0172] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0173] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), disk or CD-ROM and other media that can store program codes.

[0174] In summary, the automatic question-answering method, device, electronic device and storage medium provided by the present invention first obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful; then when the match is unsuccessful, the user questions are input into a pre-trained intent distribution model to generate intent recognition results, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question; then, based on the knowledge tool, a retrieval result corresponding to the user question is generated, and a first prompt word is constructed based on the retrieval result; finally, based on the first prompt word, the answer is determined through a pre-configured reasoning model. It can achieve 100% coverage of group finance products, 50% manual replacement rate, realize intelligent question answering, and improve the accuracy of answers. Compared with traditional RAG question and answer, the accuracy is increased by more than 20%. In addition, it improves the efficiency of knowledge use and is compatible with tables, graphs, pictures, QA and other knowledge types. In addition, all architectures of the present invention are spliced ​​in the form of modules. For new needs or problems, only adjustments need to be made to each sub-module. At the same time, the heaviest reasoning module does not require training and fine-tuning, saving a lot of resources.

[0175] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An automatic question-answering method, characterized in that: The steps include: Obtain user questions, match the user questions based on a preset knowledge base, and generate answers when the match is successful; When the match is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question; Based on the knowledge tool, generating a search result corresponding to the user question, and constructing a first prompt word based on the search result; Based on the first prompt word, an answer is determined by a pre-configured reasoning model.

2. The automatic question-answering method according to claim 1, characterized in that: The obtaining of the user question, matching the user question based on a preset knowledge base, and generating an answer when the match is successful, includes: Obtaining user questions, and inputting the user questions into the question-answering system of the knowledge base; Using a preset matching model to match the user question to obtain a matching result; Based on the matching result, it is determined whether the match is successful, and when the match is successful, an answer is generated.

3. The automatic question-answering method according to claim 2, characterized in that: The determining whether the match is successful based on the matching result includes: A similarity threshold is obtained, and the matching result is compared with the similarity threshold. When the matching result is not less than the similarity threshold, it is determined that the matching is successful; otherwise, it is determined that the matching fails.

4. The automatic question-answering method according to claim 1, characterized in that: The intention distribution model is established based on thought chain data, the input of the intention distribution model is user questions, and the output of the intention distribution model is a knowledge tool.

5. The automatic question-answering method according to claim 1, characterized in that: When the matching is unsuccessful, the user question is input into a pre-trained intent distribution model to generate an intent recognition result, including: When the match is unsuccessful, generating a second prompt word based on the user question; Determining whether an answer can be generated according to the second prompt word through the intention distribution model, and if so, directly generating the answer through the intention distribution model; When the answer cannot be generated directly, the knowledge tool name and the corresponding knowledge tool calling command are output through the intention distribution model.

6. The automatic question-answering method according to claim 1, characterized in that: The step of generating a search result corresponding to the user question based on the knowledge tool and constructing a first prompt word based on the search result includes: Based on the intention recognition result, retrieve the knowledge tool corresponding to the intention recognition result to search the user question to obtain a search result; The search result is processed to obtain a first prompt word.

7. The automatic question-answering method according to claim 1, characterized in that: The intention distribution model is the ChatGLM-6B model, and the reasoning model is the Qwen-72B model.

8. An automatic question-answering device, characterized in that: include: The knowledge base matching module is used to obtain user questions, match the user questions based on the preset knowledge base, and generate answers when the match is successful; An intent recognition module, which is used to input the user question into a pre-trained intent distribution model to generate an intent recognition result when the match is unsuccessful, wherein the intent distribution model is adapted to the scenario corresponding to the user question, and the intent recognition result is a knowledge tool or answer corresponding to the user question; A prompt word construction module, used to generate a search result corresponding to the user question based on the knowledge tool, and construct a first prompt word based on the search result; The reasoning module is used to determine the answer based on the first prompt word through a pre-configured reasoning model.

9. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer program executable by the processor; When the processor executes the computer program, the steps in the automatic question answering method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: include: Processor and memory; The memory stores a computer program executable by the processor; When the processor executes the computer program, the steps in the automatic question answering method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Intelligent interaction device and method for medical question and answer scene

    CN119397000A

  • Question answering method based on intention recognition, electronic equipment and storage medium

    CN121542398A