Industry knowledge question-answering method and system based on self-adaption and retrieval enhancement generation

Abstract

The invention relates to an industry knowledge question-answering method and system based on self-adaption and retrieval enhancement generation, and the method comprises the steps: obtaining input questions, employing a question classifier to judge the types of the input questions, and enabling the types of the input questions to comprise viewpoint type questions, data type questions, and other types of questions; respectively adopting different retrieval strategies for viewpoint questions, data questions and other questions, and outputting question answers based on a large model; and a scoring and rewriting mechanism is introduced to score and rewrite the answers to the questions based on the large model, and final answers to the questions are output. According to the method, the user questions are classified, different retrieval methods are used for each class of questions, and scoring and rewriting mechanisms are introduced, so that the knowledge retrieval efficiency is improved, the computing resources required by retrieval are reduced, and the question and answer accuracy is improved; the problems that knowledge base information is accurate but incomplete, the accuracy of internet information is insufficient, and knowledge conflicts possibly exist between the knowledge base and the internet information in an industry knowledge question and answer scene can be solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an industry knowledge question-answering method and system based on self-adaptation and retrieval-enhanced generation. Background Art

[0002] Retrieval-Augmented Generation (RAG) is a method that combines information retrieval and text generation. It assists text generation by retrieving relevant information from a knowledge base, thereby improving the relevance and accuracy of the generated text. By retrieving relevant information from external datasets, RAG effectively enhances the credibility of large models in professional fields or knowledge-intensive tasks, supports tracing the information in the answers of large models, makes the reasoning process of large models more transparent, and reduces the occurrence of "hallucination" problems of large models. Compared with large model fine-tuning technology, RAG facilitates continuous knowledge updating and improves the practicality and applicability of large models.

[0003] Retrieval-enhanced generation technology originated when the Transformer framework emerged. The classic retrieval-enhanced generation method uses three basic steps: indexing, retrieval, and generation. With the emergence of large language models, retrieval-enhanced generation adds additional steps to provide more accurate information for large models to answer complex professional or knowledge-intensive questions.

[0004] However, the current retrieval-enhanced generation still has shortcomings. The quality of the answer depends heavily on the quality of the retrieval. If the retrieval fails to return the answer or returns the wrong answer, the large model cannot generate the correct answer. At the same time, if the large model returns a lot of irrelevant information, it may also reduce the quality of the large model's answer. At the same time, in some complex scenarios or resource-intensive tasks, the retrieval-enhanced generation system often requires more computing resources. How to balance the generation effect and computing resource consumption is also known as the challenge of the current retrieval-enhanced generation system.

[0005] The current search enhancement generation technology can integrate knowledge from knowledge bases or the Internet into large models, which can effectively improve the accuracy and credibility of answers to knowledge question-answering tasks. However, in industry research knowledge question-answering tasks, using knowledge bases for retrieval will result in information loss. The large model cannot answer questions that are not in the knowledge base. Using the Internet for retrieval cannot fully utilize the knowledge in the knowledge base with strong professionalism and high accuracy. The knowledge on the Internet has not been manually reviewed, and the possibility of errors increases. Using knowledge bases and the Internet for retrieval at the same time will cause information confusion. For content with high subjectivity such as market analysis and credit recommendations, the information in the knowledge base and the Internet may conflict, thereby reducing the quality of the large model's answers. Summary of the invention

[0006] In order to address the shortcomings of the prior art, the present invention classifies user questions, uses different retrieval methods for each type of question, and introduces a scoring and rewriting mechanism, which improves the efficiency of knowledge retrieval, reduces the computing resources required for retrieval, and improves the accuracy of question and answer. It can solve the problems of accurate but incomplete knowledge base information, insufficient accuracy of Internet information, and possible knowledge conflicts between knowledge base and Internet information in industry knowledge question and answer scenarios.

[0007] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0008] An industry knowledge question answering method based on adaptive and retrieval-enhanced generation, characterized by comprising:

[0009] S1. Obtain an input question and use a question classifier to determine the category of the input question, where the category of the input question includes opinion questions, data questions, and other questions;

[0010] S2. Use different search strategies for opinion questions, data questions, and other types of questions, and output answers based on the big model;

[0011] S3. Introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

[0012] Furthermore, the construction of the question classifier includes:

[0013] Collect user questions;

[0014] Use the big model to filter out subjective questions from user questions and mark them as opinion questions;

[0015] Use the big model to filter out questions about data analysis from user questions and mark them as data-related questions;

[0016] Answer user questions using the retrieval enhancement generation method based on the self-developed industry research report knowledge base, score the answer to each user question using a large model, and mark the user questions corresponding to high-scoring answers as opinion questions;

[0017] Answer user questions based on the Internet using a search-enhanced generation method, use a large model to score the answer to each user question, and mark the user questions corresponding to high-scoring answers as data-type questions;

[0018] The high score refers to a score exceeding a preset threshold;

[0019] Label the remaining questions as Other questions.

[0020] Furthermore, different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including:

[0021] For opinion-related questions, we use vector similarity and keyword matching to obtain relevant fragments and corresponding relevance from the self-developed industry research report knowledge base. We use top-k sampling, set the k value, and take out the k relevant fragments with the highest relevance. We input the k relevant fragments and the question into the big model and output the answer to the question.

[0022] Furthermore, different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including:

[0023] For data-related problems, we use vector similarity and keyword matching to retrieve relevant fragments and their corresponding relevance in the self-developed industry research report knowledge base, use top-k and top-p to sample the search results, set the k value, and extract the k relevant fragments with the highest relevance;

[0024] Set the relevance of the segment to the question to c i ∈[0,1], (where i is an integer,) the sum of the correlations of the k related segments is

[0025] Set the parameter p, if There is no need to use the Internet to search for additional information;

[0026] like Then, information is retrieved from the Internet, and the retrieved web pages are sorted from high to low according to their relevance to the question;

[0027] Set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that and

[0028] It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n;

[0029] The k related fragments, n related web pages and questions are input into the big model, and the big model outputs answers to the questions based on the retrieved information.

[0030] Furthermore, different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including:

[0031] For other types of questions, use the Internet to retrieve information, use top-p for sampling, and sort the web pages retrieved from the Internet from high to low according to relevance;

[0032] Set the parameter p and set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that

[0033] It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n;

[0034] The n related web pages and questions are input into the big model, and the big model generates answers to the questions based on the retrieved information.

[0035] Furthermore, the method further comprises:

[0036] Set the score threshold θ and the maximum number of rewrites q for the question answer max , the answer to the question and the corresponding question are input into the large model for scoring, and the score s0 of the initial answer O0 is obtained;

[0037] When the score s0 is not less than the score threshold θ, the initial answer O0 is directly returned;

[0038] When the score s0 is less than the score threshold θ, the first answer O0 is rewritten using the large model, and a search is performed based on the rewritten question answer, and the large model outputs the first rewritten answer O1; the first rewritten answer O1 of the large model is scored to obtain the score s1 of the first rewritten answer O1; if the score s1 is not less than the score threshold θ, the first rewritten answer O1 is directly returned; if the score s1 is less than the score threshold θ, the above steps are repeated;

[0039] If the current rewrite count i is equal to the maximum rewrite count q max , and the answer is O for the i-th rewrite i Scores i If the score is less than the threshold value θ, the rewriting is stopped and the answer {O0, O1…O i The highest-scoring answer among} is taken as the answer to the question.

[0040] The present invention also relates to an industry knowledge question-answering system based on adaptive and retrieval-enhanced generation, characterized in that it includes:

[0041] A classification module is used to obtain input questions and use a question classifier to determine the category of the input questions, where the categories of the input questions include opinion questions, data questions, and other questions;

[0042] The retrieval module is used to adopt different retrieval strategies for opinion questions, data questions, and other types of questions, and output answers to the questions based on the big model;

[0043] The rewriting module is used to introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

[0044] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0045] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0046] The memory is used to store program instructions;

[0047] The processor is used to execute the above method by calling program instructions.

[0048] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0049] The beneficial effects of the present invention are:

[0050] The present invention proposes an industry knowledge question-and-answer method and system based on adaptive and retrieval enhancement generation. In view of the characteristics of strong professionalism, complex information sources, and subjectiveness of some knowledge in industry knowledge question-and-answer, user questions are divided into three categories according to the content of the questions, and different retrieval methods are used for each type of question. This scheme not only improves the efficiency of knowledge retrieval and reduces the computing resources required for retrieval, but also solves the problem that the knowledge in the knowledge base is accurate but not comprehensive and the knowledge in the Internet is rich but inaccurate, and utilizes the advantages of knowledge base and Internet retrieval to improve the effect of knowledge question-and-answer. At the same time, the scheme also reduces the impact of knowledge conflict between the Internet and the knowledge base on the question-and-answer effect, improves the accuracy of question-and-answer, makes full use of the external knowledge of the Internet, and expands the scope of large model question-and-answer. The accuracy of the question-and-answer of the present invention is about 92%, which is about 20% higher than that of the traditional retrieval enhancement generation method, and the maximum delay time is about 5% higher than that of the traditional retrieval enhancement generation method. At the same time, the utilization rate of computing resources is improved. The present invention also adds a scoring and rewriting mechanism, which can further improve the effect of question-and-answer when computing resources are sufficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0052] Figure 1 A schematic diagram of the process flow of the industry knowledge question-answering method based on adaptive and retrieval-enhanced generation according to the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of the industry knowledge question-answering system based on adaptive and retrieval-enhanced generation according to the present invention;

[0054] Figure 3 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments, and these embodiments are only used to illustrate the present application and are not used to limit the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0056] By understanding the development trends, risks and opportunities of different industries through industry research, we can better serve customers, manage risks and formulate business strategies. As digital transformation continues to deepen, artificial intelligence can provide more accurate and efficient services. The industry knowledge question-and-answer system uses large models to achieve accurate, professional and instant answers to user questions, promote the intelligent acquisition of industry knowledge, and help quickly grasp industry trends and key insights.

[0057] like Figure 1 As shown, the embodiment of the present invention discloses an industry knowledge question answering method based on adaptive and retrieval enhanced generation, and the steps of the process include:

[0058] Step S1. Obtain input questions, and use a question classifier to determine the categories of the input questions, where the categories of the input questions include opinion questions, data questions, and other questions.

[0059] In this embodiment, the knowledge base used by the adaptive hybrid retrieval enhanced generation industry research intelligent question-answering system is a self-developed industry research report knowledge base, which contains industry overviews, market analysis, main industry chains and key enterprise analysis, profit models, industry risks, credit recommendations, risk control points, etc. for various industries such as new energy, real estate, and medical care. Documents in the knowledge base are divided into blocks according to content to increase the accuracy of retrieval.

[0060] Specifically, the question classifier in this embodiment is a shallow neural network, the input is a vector corresponding to the user input question converted by the text embedding model, and the output is a question category identifier. The construction of the question classifier includes:

[0061] Collect user questions;

[0062] Use the big model to filter out subjective questions from user questions and mark them as opinion questions;

[0063] Use the big model to filter out questions about data analysis from user questions and mark them as data-related questions;

[0064] Answer user questions using the retrieval enhancement generation method based on the self-developed industry research report knowledge base, score the answer to each user question using a large model, and mark the user questions corresponding to high-scoring answers as opinion questions;

[0065] Answer user questions based on the Internet using a search-enhanced generation method, use a large model to score the answer to each user question, and mark the user questions corresponding to high-scoring answers as data-type questions;

[0066] The high score refers to a score exceeding a preset threshold;

[0067] Label the remaining questions as Other questions.

[0068] For each question input by the user, the question classifier is first used to judge the category of the question. The question classifier is a small model trained with a data set. After the question is input, it can quickly give the category corresponding to each question. The data set used for training can be constructed using manual annotation or large model annotation methods. According to the content of the question, the question classifier divides user questions into three categories: "opinion", "data", and "other". Opinion questions are questions about subjective judgments on market forms and service solutions, such as "What is the future trend of the real estate market?" and "What is the credit recommendation for companies in the artificial intelligence industry?"; data questions are questions about objective data and situations in the industry, such as "China's photovoltaic production capacity in 2023" and "Revenue of leading companies in the new energy industry"; other questions are questions other than opinion and data questions, such as general knowledge questions such as "What is photovoltaic?"

[0069] Step S2. Use different retrieval strategies for opinion questions, data questions, and other types of questions, and output answers to the questions based on the big model.

[0070] Specifically, for opinion questions, we use vector similarity and keyword matching to obtain relevant fragments and corresponding relevance from the self-developed industry research report knowledge base, use top-k sampling, set the k value, take out the k relevant fragments with the highest relevance, input the k relevant fragments and questions into the big model, and output the answer to the question. For opinion questions, since there may be different opinions on the same thing on the Internet, these opinions may be contrary to the opinions in the self-developed industry research reports, and it is difficult to screen the Internet opinions, and the quality of these opinions cannot be guaranteed. Therefore, the answer to opinion questions comes from the self-developed industry research report knowledge base, which is used to search and sample through top-k. Using vector similarity and keyword matching methods, we take out the k fragments with the highest relevance from the self-developed industry research report knowledge base, and input these k relevant fragments and questions into the big model. The big model outputs the answer based on the k relevant fragments, and uses the advantages of the self-developed industry research report knowledge base to improve the effect of knowledge question and answer, and reduce the impact of knowledge conflicts between the Internet and the knowledge base on the question and answer effect.

[0071] For data-related questions, the data required for some questions can be directly obtained from self-developed industry reports, the data required for some questions are not mentioned in the industry reports and need to be retrieved on the Internet, and some questions require a combination of self-developed industry reports and data from the Internet to obtain richer answers. Therefore, for data-related questions, a hybrid retrieval method of knowledge base and Internet is used, and the retrieved results are sampled using a hybrid method of top-k and top-p.

[0072] Specifically, for data-related problems, we use vector similarity and keyword matching to retrieve relevant fragments and their corresponding relevance in the self-developed industry research report knowledge base, use top-k and top-p to sample the search results, set the k value, and extract the k relevant fragments with the highest relevance;

[0073] Set the relevance of the segment to the question to c i ∈[0,1], (where i is an integer,) the sum of the correlations of the k related segments is

[0074] Set the parameter p, if There is no need to use the Internet to search for additional information;

[0075] like Then, information is retrieved from the Internet, and the retrieved web pages are sorted from high to low according to their relevance to the question;

[0076] Set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that And m is as small as possible, m satisfies

[0077] It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n;

[0078] The k related fragments, n related web pages and questions are input into the big model, and the big model outputs the answer to the question based on the retrieved information. The above-mentioned knowledge base and Internet hybrid retrieval method is flexible and can dynamically adjust the number of web pages retrieved after the Internet search based on the relevance of the knowledge in the self-developed industry research report knowledge base and the question, ensuring the richness of the input information, improving the accuracy of the answer, reducing the interference of irrelevant information on the answer generated by the big model, avoiding unnecessary waste of computing resources, and improving the efficiency of system operation.

[0079] For other types of questions, we retrieve information through the Internet and the internal knowledge of the big model obtained during the pre-training of the big model. Since the knowledge on the Internet has not been manually screened, there is an uneven distribution of knowledge. Therefore, we use top-p for sampling and sort the web pages retrieved from the Internet from high to low according to relevance.

[0080] Set the parameter p and set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that

[0081] It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n;

[0082] The n related web pages and questions are input into the big model, and the big model generates answers to the questions based on the retrieved information.

[0083] In this embodiment, m should be as small as possible. At the same time, since the input length of the large model is limited, too much input will increase the usage of computing resources, so the number of desirable related web pages is limited.

[0084] In this embodiment, the self-developed industry research report knowledge base includes self-developed industry research reports, which are segmented into blocks according to semantics using a large model, and an open source text embedding model (Embedding model), such as BAAI GeneralEmbedding-Large-zh (Bge-large-zh), is used to calculate the vector of the fragment.

[0085] Step S3. Introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

[0086] Specifically, set the score threshold θ and the maximum number of rewrites q for the question answer max , the answer to the question and the corresponding question are input into the large model for scoring, and the score s0 of the initial answer O0 is obtained;

[0087] When the score s0 is not less than the score threshold θ, the initial answer O0 is directly returned;

[0088] When the score s0 is less than the score threshold θ, the first answer O0 is rewritten using the large model, and a search is performed based on the rewritten question answer, and the large model outputs the first rewritten answer O1; the first rewritten answer O1 of the large model is scored to obtain the score s1 of the first rewritten answer O1; if the score s1 is not less than the score threshold θ, the first rewritten answer O1 is directly returned; if the score s1 is less than the score threshold θ, the above steps are repeated;

[0089] If the current rewrite count i is equal to the maximum rewrite count q max , and the answer is O for the i-th rewrite i Scores i If the score is less than the threshold value θ, the rewriting is stopped and the answer {O0, O1…O i The highest-scoring answer among} is taken as the answer to the question.

[0090] In the embodiment of the present invention, the big model may use an open source general big model such as Tongyi Qianwen, or a closed source big model.

[0091] like Figure 2 As shown, the embodiment of the present invention discloses an industry knowledge question-answering system based on adaptive and retrieval-enhanced generation, and its structure includes:

[0092] A classification module is used to obtain input questions and use a question classifier to determine the category of the input questions, where the categories of the input questions include opinion questions, data questions, and other questions;

[0093] The retrieval module is used to adopt different retrieval strategies for opinion questions, data questions, and other types of questions, and output answers to the questions based on the big model;

[0094] The rewriting module is used to introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

[0095] By using this system, the above-mentioned operation processing method can be executed and the corresponding technical effects can be achieved.

[0096] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps in the method in the above embodiment, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the method in the above embodiment are implemented.

[0097] The embodiment of the present invention further provides an electronic device for executing the above method, as a device for implementing the method, and its structure is as follows: Figure 3 As shown, the electronic device 300 includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one. It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the specific structure of the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. Optionally, the electronic device may also include a display screen 305 for displaying images, or receiving user operation instructions when necessary.

[0098] The processor 301 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 304 may include a receiver and a transmitter. The transceiver 304 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0099] Processor 301 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0100] The processor 301 may also include one or more processing units, for example, the processor 301 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 300. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 301 for storing instructions and data. In some embodiments, the memory in the processor 301 is a high-speed cache memory. The memory may store instructions or data that the processor 301 has just used or circulated. If the processor 301 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 301 is reduced, and the efficiency of the system is improved.

[0101] The processor 301 can run the method provided in the embodiment of the present application. The processor 301 can include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU can cooperate to execute the method provided in the embodiment of the present application, for example, part of the algorithm in the method is executed by the CPU, and another part of the algorithm is executed by the GPU to obtain faster processing efficiency.

[0102] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or may include a high-speed random access memory, or may include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0104] Optionally, the memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the method provided in any embodiment of the present application.

[0105] The memory 303 can be used to store computer executable program codes, which include instructions. The processor 301 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the memory 303. The memory 303 may include a program storage area and a data storage area. Among them, the program storage area may store the code of the operating system, application program, etc. The data storage area may store data created during the use of the electronic device 300 (such as images, videos, etc. collected by the camera application), etc.

[0106] The memory 303 may also store one or more computer programs corresponding to the method provided in the embodiment of the present application. The one or more computer programs are stored in the above-mentioned memory 303 and are configured to be executed by the one or more processors 301. The one or more computer programs include instructions, and the above-mentioned instructions can be used to execute each step in the above-mentioned corresponding embodiment.

[0107] Of course, the code of the method provided in the embodiment of the present application can also be stored in an external memory. In this case, the processor 301 can run the code of the method stored in the external memory through the external memory interface, and the processor 301 can control the running process.

[0108] The display screen 305 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include 1 or N display screens 305, where N is a positive integer greater than 1. The display screen 305 can be used to display information input by a user or information provided to a user and various graphical user interfaces (GUI). For example, the display screen 305 can display photos, videos, web pages, or files.

[0109] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0110] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0112] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0114] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An industry knowledge question answering method based on adaptive and retrieval-enhanced generation, characterized in that: include: S1. Obtain an input question and use a question classifier to determine the category of the input question, where the category of the input question includes opinion questions, data questions, and other questions; S2. Use different search strategies for opinion questions, data questions, and other types of questions, and output answers based on the big model; S3. Introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

2. The method according to claim 1, characterized in that The construction of the question classifier includes: Collect user questions; Use the big model to filter out subjective questions from user questions and mark them as opinion questions; Use the big model to filter out questions about data analysis from user questions and mark them as data-related questions; Answer user questions using the retrieval enhancement generation method based on the self-developed industry research report knowledge base, score the answer to each user question using a large model, and mark the user questions corresponding to high-scoring answers as opinion questions; Answer user questions based on the Internet using a search-enhanced generation method, use a large model to score the answer to each user question, and mark the user questions corresponding to high-scoring answers as data-type questions; The high score refers to a score exceeding a preset threshold; Label the remaining questions as Other questions.

3. The method according to claim 1, characterized in that Different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including: For opinion-related questions, we use vector similarity and keyword matching to obtain relevant fragments and corresponding relevance from the self-developed industry research report knowledge base. We use top-k sampling, set the k value, and take out the k relevant fragments with the highest relevance. We input the k relevant fragments and the question into the big model and output the answer to the question.

4. The method according to claim 1, characterized in that Different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including: For data-related problems, we use vector similarity and keyword matching to retrieve relevant fragments and their corresponding relevance in the self-developed industry research report knowledge base, use top-k and top-p to sample the search results, set the k value, and extract the k relevant fragments with the highest relevance; Set the relevance of the segment to the question to c i ∈[0,1], (where i is an integer,) the sum of the correlations of the k related segments is Set the parameter p, if There is no need to use the Internet to search for additional information; like Then, information is retrieved from the Internet, and the retrieved web pages are sorted from high to low according to their relevance to the question; Set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that and It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n; The k related fragments, n related web pages and questions are input into the big model, and the big model outputs answers to the questions based on the retrieved information.

5. The method according to claim 1, characterized in that Different search strategies are used for opinion questions, data questions, and other types of questions, and answers to the questions are output based on the big model, including: For other types of questions, use the Internet to retrieve information, use top-p for sampling, and sort the web pages retrieved from the Internet from high to low according to relevance; Set the parameter p and set the relevance of the web page to the question to r i ∈[0,1], take the first m web pages with high relevance, so that It is assumed that the first n relevant web pages are taken from the first m web pages, m≥n; The n related web pages and questions are input into the big model, and the big model generates answers to the questions based on the retrieved information.

6. The method according to any one of claims 3 to 5, characterized in that: The method further comprises: Set the score threshold θ and the maximum number of rewrites q for the question answer max , the answer to the question and the corresponding question are input into the large model for scoring, and the score s0 of the initial answer O0 is obtained; When the score s0 is not less than the score threshold θ, the initial answer O0 is directly returned; When the score s0 is less than the score threshold θ, the first answer O0 is rewritten using the large model, and a search is performed based on the rewritten question answer, and the large model outputs the first rewritten answer O1; the first rewritten answer O1 of the large model is scored to obtain the score s1 of the first rewritten answer O1; if the score s1 is not less than the score threshold θ, the first rewritten answer O1 is directly returned; if the score s1 is less than the score threshold θ, the above steps are repeated; If the current rewrite count i is equal to the maximum rewrite count q max , and the answer is O for the i-th rewrite i Scores i If the score is less than the threshold value θ, the rewriting is stopped and the answer {O0, O1…O i The highest-scoring answer among} is taken as the answer to the question.

7. An industry knowledge question-answering system based on adaptive and retrieval-enhanced generation, characterized in that: include: A classification module is used to obtain input questions and use a question classifier to determine the category of the input questions, where the categories of the input questions include opinion questions, data questions, and other questions; The retrieval module is used to adopt different retrieval strategies for opinion questions, data questions, and other types of questions, and output answers to the questions based on the big model; The rewriting module is used to introduce a scoring and rewriting mechanism to score and rewrite the answer to the question based on the large model and output the final answer to the question.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store program instructions; The processor is configured to execute the method according to any one of claims 1 to 6 by calling program instructions.

10. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images