Method and equipment for improving knowledge retrieval precision of specific industry

Abstract

The invention relates to the technical field of natural language processing, in particular to a method and equipment for improving the knowledge retrieval precision of a specific industry, and aims at improving the precision of retrieval answers. The method for improving the knowledge retrieval precision of the specific industry comprises the following steps: generating a question vector according to historical questions and current questions of a user; retrieving a vector database according to the problem vector, and obtaining N retrieval vectors with the highest accuracy; solving N target text blocks corresponding to the N retrieval vectors according to a mapping relation between the knowledge text blocks and the vector representation; according to the N target text blocks and the first prompt word, generating a preliminary answer based on a first large language model; and according to the historical question, the current question, the second cue word and the preliminary answer, generating a final answer based on a second large language model. According to the method, a plurality of links such as problem generation, retrieval and result optimization are improved, so that the knowledge retrieval of a specific industry can be effectively improved, and the generation precision can be enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and device for improving the accuracy of knowledge retrieval in a specific industry. Background Art

[0002] In the era of information explosion, all industries are faced with the problem of processing and utilizing massive amounts of data. Especially in specific industries, the accuracy of knowledge retrieval is directly related to the competitiveness and innovation ability of enterprises.

[0003] Traditional search engines are mainly based on keyword matching and full-text retrieval technology. These methods perform well when dealing with general information retrieval. However, in specific industries, due to the complexity of professional terminology, simple keyword matching often leads to the emergence of a large amount of irrelevant information, while truly relevant information may be omitted, making the retrieval results sometimes unable to meet user needs. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention proposes a method and a device for improving the accuracy of retrieval of specific industry knowledge, thereby improving the accuracy of retrieval answers.

[0005] A first aspect of the present invention provides a method for improving the accuracy of retrieval of specific industry knowledge, the method comprising: Generate a question vector based on the user's historical questions and current questions; According to the question vector, a vector database is searched to obtain N search vectors with the highest accuracy; wherein the vector database stores a number of vector representations generated by knowledge text blocks of the specific industry; According to the mapping relationship between the knowledge text blocks and the vector representations, N target text blocks corresponding to the N search vectors are obtained; Generate a preliminary answer based on the first language model according to the N target text blocks and the first prompt word; Generate a final answer based on the second largest language model according to the historical question, the current question, the second prompt word and the preliminary answer; Wherein, N is a preset value.

[0006] Preferably, the step of "generating a question vector according to the user's historical questions and current questions" includes: Generate a question to be answered based on the third language model according to the historical question, the current question and the third prompt word; The question to be answered is converted into the question vector.

[0007] Preferably, the step of “searching a vector database according to the question vector and obtaining N search vectors with the highest accuracy” includes: According to the question vector, the vector database is searched using a cosine similarity algorithm to obtain M search vectors, which then form a first vector set; According to the question vector, the vector database is searched using the MMR (Maximal Marginal Relevance) algorithm to obtain M search vectors, which then form a second vector set; Obtaining the intersection of the first vector set and the second vector set; Filtering and rearranging the search vectors in the intersection, and selecting the N search vectors with the highest accuracy; Wherein, M is a preset value.

[0008] Preferably, the method further comprises: A plurality of the knowledge text blocks are generated according to the file database of the specific industry, and the knowledge text blocks are converted into vector representations and stored in the vector database.

[0009] Preferably, the step of "generating a plurality of the knowledge text blocks according to the file database of the specific industry, and converting the knowledge text blocks into vector representations and storing them in the vector database" includes: For each file in the file database of the specific industry, read the text in the file and interpret the pictures in the file to generate a text file; Segment each of the text files to obtain a number of original text blocks that do not exceed a preset length; Performing text cleaning on the original text block based on the fourth language model to obtain the knowledge text block; Convert each of the knowledge text blocks into the corresponding vector representation; The mapping relationship between the knowledge text block and the vector representation is saved, and the vector representation is stored in the vector database.

[0010] Preferably, the step of "segmenting each of the text files to obtain a plurality of original text blocks not exceeding a preset length" includes: Splitting each of the text files according to the first-level headings to obtain a plurality of first text blocks; Determine the length of each of the first text blocks, and if the length of a first text block exceeds the preset length, divide the first text block according to the secondary headings to obtain a plurality of second text blocks; Determine the length of each second text block, and if the length of a second text block exceeds the preset length, segment the second text block according to the third-level headings to obtain a plurality of third text blocks; Determine the length of each of the third text blocks, and if the length of a third text block exceeds the preset length, segment the third text block into paragraphs to obtain a plurality of fourth text blocks; Determine the length of each of the fourth text blocks, and if the length of a fourth text block exceeds the preset length, segment the fourth text block according to the period to obtain a plurality of fifth text blocks; Merging the adjacent fifth text blocks that are derived from the same fourth text block in order to obtain a plurality of sixth text blocks; each of the sixth text blocks contains one or more fifth text blocks, and the length does not exceed the preset length; Constructing a set of the original text blocks; The original text blocks include: the first text blocks, the second text blocks, the third text blocks, the fourth text blocks, the fifth text blocks that have not been merged, and the sixth text blocks, all of which have lengths not exceeding the preset length.

[0011] Preferably, the method further comprises: Constructing a data set for the specific industry, wherein the data set includes a first preset number of question samples and corresponding standard answers; Dividing the data set into a training set and a test set according to a preset ratio; Using the training set, fine-tuning a second preset number of large language models respectively using LoRA (Low-Rank Adaptation) technology; According to the input of the administrator, the first large language model, the second large language model, the third large language model and the fourth large language model are respectively selected from the second preset number of fine-tuned large language models, and the first large language model, the second large language model, the third large language model and the fourth large language model are the same or different.

[0012] Preferably, the first prompt word is a preset prompt word, and the second prompt word and the third prompt word are preset prompt words or prompt words set by the administrator; The method further comprises: Using the test set to score the retrieval and generation effects based on the RAGAS (Retrieval Augmented Generation Assessment) standard, and obtaining a scoring result; The scoring results include: answer relevance, fidelity and context relevance.

[0013] According to a second aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method described above.

[0014] According to a third aspect of the present invention, a computer-readable storage device is provided, storing a computer program that can be loaded by a processor and execute the method described above.

[0015] The present invention has the following beneficial effects: The method for improving the retrieval accuracy of specific industry knowledge proposed in the present invention searches the vector database according to the question vector and obtains the N retrieval vectors with the highest accuracy, then finds the corresponding N target text blocks according to the mapping relationship, and then uses the first largest language model to refine and summarize the target text blocks to obtain a preliminary answer, and finally uses the second largest language model to optimize the preliminary answer to obtain the final answer. Through the above-mentioned retrieval, obtaining the N retrieval vectors with the highest accuracy, refining, summarizing and optimizing the text blocks, the present invention improves the accuracy of the retrieval results step by step, so that the final answer can well fit the question raised by the user.

[0016] The cosine similarity algorithm measures similarity by calculating the cosine value of the angle between vectors, which is suitable for measuring the similarity in the direction of vectors; while the MMR algorithm comprehensively considers the relevance and diversity of the results, which helps to avoid redundancy and repetition. Combining the two can make full use of their respective advantages and improve the accuracy of retrieval results. By finding the intersection of the two retrieval results, we can further filter out those vectors that only meet the standards of a single algorithm but are not comprehensive enough, leaving those vectors that meet the standards of both algorithms at the same time. These vectors are often more representative in terms of similarity and diversity. The vectors in the intersection are filtered and rearranged, and then the N retrieval vectors with the highest accuracy are selected, so that the N retrieval vectors obtained are often more balanced in terms of similarity and diversity.

[0017] The text block segmentation method of the present invention ensures that the generated original text block set contains all the sentences in the text file without duplication.

[0018] Large language models are used when building vector databases, generating questions to be answered, generating preliminary answers, and generating final answers. These large language models are fine-tuned using industry-specific data sets, allowing them to better adapt to the characteristics of relevant knowledge in that specific industry during their working process, further improving the accuracy of the final answer.

[0019] In summary, the present invention has made improvements in multiple aspects including database construction, question generation, database retrieval, result screening, text extraction, summarization and optimization, which can effectively improve the accuracy of knowledge retrieval and enhancement generation in specific industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the main steps of Embodiment 1 of the method for improving the retrieval accuracy of specific industry knowledge in the present invention; Figure 2 This is a schematic diagram of the main steps of Embodiment 2 of the method for improving the retrieval accuracy of specific industry knowledge in the present invention; Figure 3 It is a schematic diagram of the main steps of Example 3 of the method for improving the retrieval accuracy of specific industry knowledge in the present invention. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that, in the description of the present invention, the terms "first" and "second" are only for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore cannot be understood as limiting the present invention. In addition, the term "and / or" in the present invention is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0024] Figure 1 Schematic diagram of the main steps of the first embodiment of the method for improving the retrieval accuracy of specific industry knowledge in the present invention. Figure 1 As shown, the method of this embodiment includes steps A10-A50: Step A10: Generate a question vector based on the user's historical questions and current questions. This step may specifically include steps A11-A12: Step A11: Generate questions to be answered based on the third language model according to the historical questions, the current questions and the third prompt word.

[0025] For example, the previous question entered by the user is: How much does the Class A auto insurance launched by Company A cost per year? The current question entered is: OK, thank you! Another question, what does this auto insurance cover? The third prompt word is: Combine the previous question and the current question, remove redundant information, and accurately understand the user's true intention. The generated question to be answered is: What does the Class A auto insurance launched by Company A cover? Step A12: Convert the question to be answered into a question vector.

[0026] In this embodiment, BGE is used to convert text into vectors.

[0027] BGE (BGEmbeddings) is a powerful text vectorization tool developed by the BAAI team. It can efficiently convert Chinese text into numerical vectors and is widely used in NLP tasks such as text retrieval, classification, and clustering.

[0028] Step A20: Search the vector database according to the question vector and obtain the N search vectors with the highest accuracy. The vector database stores a number of vector representations generated by knowledge text blocks of a specific industry.

[0029] This step may specifically include steps A21-A24: Step A21: According to the question vector, a cosine similarity algorithm is used to search the vector database to obtain M search vectors, which then form a first vector set.

[0030] Cosine Similarity is a method to measure the similarity between two vectors in space. It determines the similarity between two vectors by calculating the cosine value of the angle between the two vectors. It focuses on the direction of the two vectors rather than their magnitude, so it can well capture the similarity between vectors without being affected by the length of the vectors. The value of cosine similarity ranges from -1 to 1, where 1 means that the vectors are exactly the same, 0 means there is no correlation between the two vectors, and -1 means they are completely opposite.

[0031] Step A22: According to the question vector, the vector database is searched using the MMR algorithm to obtain M search vectors, which then form a second vector set, where M is a preset value.

[0032] MMR (Maximal Marginal Relevance) is a commonly used technology in information retrieval and text summary generation. MMR retrieval aims to balance the relevance and diversity of search results. Its core idea is to increase the overall diversity by selecting documents or fragments that are dissimilar to the selected results while maintaining the relevance of the results, thereby improving the coverage of information and user satisfaction with the search results.

[0033] Step A23: Obtain the intersection of the first vector set and the second vector set.

[0034] Step A24: filter and rearrange the search vectors in the intersection, and select N search vectors with the highest accuracy, where N is a preset value and M>N.

[0035] Step A30: According to the mapping relationship between the knowledge text blocks and the vector representations, N target text blocks corresponding to the N search vectors are obtained.

[0036] Specifically, when the vector database is pre-built, the mapping relationship between the knowledge text blocks and the vector representation is also stored. Based on this mapping relationship, the N knowledge text blocks corresponding to the N search vectors can be found, which are called target text blocks.

[0037] Step A40: Generate a preliminary answer based on the first language model according to the N target text blocks and the first prompt word.

[0038] In this embodiment, the first prompt word may be a prompt word preset in the program, for example: extracting and summarizing key information according to the input target text block.

[0039] Step A50: Generate a final answer based on the second largest language model according to the historical question, the current question, the second prompt word and the preliminary answer.

[0040] In this embodiment, the second prompt word is a prompt word preset in the program or a prompt word set by the administrator of the industry knowledge retrieval system. For example: Please analyze the user's exact intention based on the user's historical questions and current questions, and generate a standard, highly accurate, and less complicated answer based on the input preliminary answer.

[0041] Figure 2 Schematic diagram of the main steps of the second embodiment of the method for improving the retrieval accuracy of specific industry knowledge in the present invention. Figure 2 As shown, the method of this embodiment includes steps B10-B60: Step B10: Generate a number of knowledge text blocks based on the file database of a specific industry, and convert the knowledge text blocks into vector representations and store them in the vector database.

[0042] This step may specifically include steps B11-B15: Step B11: For each file in the file database of the specific industry, read the text in the file and interpret the pictures in the file to generate a text file.

[0043] In this embodiment, the files stored in the file database of a specific industry include formats such as word and pdf, which also include some pictures and tables. In this embodiment, in addition to reading the text part and the content in the table, OCR (Optical Character Recognition) and other technologies are also used to interpret the pictures. The final generated text file is in txt format.

[0044] Step B12: Segment each text file to obtain a number of original text blocks that do not exceed a preset length. This step may specifically include steps B121-B127: Step B121, split each text file according to the first-level title to obtain a plurality of first text blocks.

[0045] Step B122: determine the length of each first text block. If the length of a first text block exceeds a preset length, divide the first text block according to the secondary headings to obtain a plurality of second text blocks.

[0046] In this embodiment, the preset length is 300.

[0047] Step B123, determine the length of each second text block, if the length of a second text block exceeds the preset length, divide the second text block according to the third-level headings to obtain a plurality of third text blocks.

[0048] Step B124, determining the length of each third text block, if the length of a third text block exceeds a preset length, dividing the third text block into paragraphs to obtain a plurality of fourth text blocks.

[0049] Step B125, determining the length of each fourth text block, if the length of a fourth text block exceeds a preset length, dividing the fourth text block according to periods to obtain a plurality of fifth text blocks.

[0050] Step B126: merge the fifth text blocks that are derived from the same fourth text block and are adjacent to each other in order to obtain a plurality of sixth text blocks. Each sixth text block contains one or more fifth text blocks, and the length does not exceed a preset length.

[0051] After step B125, each fifth text block is a sentence ending with a period, and it is assumed that it will not exceed 300 words. However, if each text block is too small, it will increase the workload of the later retrieval, so it is necessary to merge in step B126, the purpose is to meet the requirement of not exceeding 300 words and to reduce the number of text blocks as much as possible.

[0052] For example, in step B125, a fourth text block is divided into 10 fifth text blocks according to the period. In step B126, it is found that the length of blocks 1-5 after merging is 243 characters, and the length of blocks 1-6 after merging is 302 characters, so blocks 1-5 are merged into a sixth text block; then the same method is used to merge blocks 6-9 into a sixth text block with a length of 293, and the remaining 10th fifth text block is no longer merged as a single block.

[0053] Step B127, construct a set of original text blocks.

[0054] The "original text block" referred to in this embodiment refers to the text block obtained after the segmentation operation of the above steps B121-B126 and has not been cleaned, including: the first text block, the second text block, the third text block, the fourth text block, the unmerged fifth text block and the sixth text block whose length does not exceed the preset length.

[0055] The text block segmentation and merging method of this embodiment ensures that the set of original text blocks contains all the sentences in the text file without duplication.

[0056] Step B13: Clean the original text block based on the fourth language model to obtain a knowledge text block.

[0057] Among them, the cleaning operation includes: removing useless content such as advertisements, garbled characters, HTML tags, etc., correcting language errors, standardizing expressions, and removing or replacing sensitive information.

[0058] Step B14: Convert each knowledge text block into a corresponding vector representation.

[0059] Step B15: Save the mapping relationship between the knowledge text block and the vector representation, and store the vector representation in the vector database.

[0060] Steps B20-B60 correspond to steps A10-A50 in the first embodiment and are the same as those in the first embodiment, and are not described again here.

[0061] Figure 3 Schematic diagram of the main steps of the third embodiment of the method for improving the retrieval accuracy of specific industry knowledge in the present invention. Figure 3 As shown, the method of this embodiment includes steps C10-C80: Step C10: Generate a number of knowledge text blocks based on the file database of a specific industry, and convert the knowledge text blocks into vector representations and store them in the vector database.

[0062] Step C20: fine-tune a second preset number of large language models respectively.

[0063] This step specifically includes steps C21-C23: Step C21: construct a data set for a specific industry, wherein the data set includes a first preset number of question samples and corresponding standard answers.

[0064] Step C22: Divide the data set into a training set and a test set according to a preset ratio.

[0065] Step C23: fine-tune the second preset number of large language models respectively using the training set and the LoRA technology.

[0066] In this embodiment, the first preset number is 30,000, and the second preset number is 6. LoRA (Low-Rank Adaptation) is a lightweight model fine-tuning training method, which is mainly used to fine-tune the model based on the original large model.

[0067] Step C30: According to the input of the administrator, a first language model, a second language model, a third language model and a fourth language model are respectively selected from the second preset number of fine-tuned large language models, and the first language model, the second language model, the third language model and the fourth language model are the same or different.

[0068] In the above step C20, after fine-tuning the six major language models using the training set, the administrator of the retrieval system can select the first major language model for generating preliminary answers, the second major language model for generating final answers, the third major language model for generating questions to be answered, and the fourth major language model for cleaning text blocks from the six major language models according to actual usage.

[0069] Steps C40-C80 correspond to steps A10-A50 in the first embodiment and are the same as those in the first embodiment, and are not described again here.

[0070] In an optional embodiment, the method further includes: using the test set to score the retrieval and generation effects based on the RAGAS standard to obtain a scoring result.

[0071] Among them, the scoring results include: answer relevance, fidelity and context relevance.

[0072] RAGAS is a framework for no-reference evaluation of Retrieval Augmented Generation (RAG) systems.

[0073] Answer relevance is used to evaluate the relevance between the generated answer and the question; faithfulness is used to evaluate the factual consistency between the generated answer and the retrieved context; and context relevancy is used to evaluate the relevance between the retrieved context and the question.

[0074] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art can understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0075] Furthermore, based on the above method embodiment, the present invention also provides an embodiment of an electronic device. The electronic device of this embodiment includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute the method as described above.

[0076] Furthermore, the present invention also provides an embodiment of a computer-readable storage device, in which the storage device of this embodiment stores a computer program that can be loaded by a processor and execute the method described above.

[0077] The computer-readable storage device may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0078] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0079] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for improving the accuracy of retrieval of specific industry knowledge, characterized in that: The method comprises: Generate a question vector based on the user's historical questions and current questions; According to the question vector, a vector database is searched to obtain N search vectors with the highest accuracy; wherein the vector database stores a number of vector representations generated by knowledge text blocks of the specific industry; According to the mapping relationship between the knowledge text blocks and the vector representations, N target text blocks corresponding to the N search vectors are obtained; Generate a preliminary answer based on the first language model according to the N target text blocks and the first prompt word; Generate a final answer based on the second largest language model according to the historical question, the current question, the second prompt word and the preliminary answer; Wherein, N is a preset value.

2. The method for improving the retrieval accuracy of specific industry knowledge according to claim 1, characterized in that: The steps of "generating a question vector based on the user's historical questions and current questions" include: Generate a question to be answered based on the third language model according to the historical question, the current question and the third prompt word; The question to be answered is converted into the question vector.

3. The method for improving the retrieval accuracy of specific industry knowledge according to claim 1, characterized in that: The step of "searching a vector database according to the question vector and obtaining N search vectors with the highest accuracy" includes: According to the question vector, the vector database is searched using a cosine similarity algorithm to obtain M search vectors, which then form a first vector set; According to the question vector, the vector database is searched using the MMR algorithm to obtain M search vectors, which then form a second vector set; Obtaining the intersection of the first vector set and the second vector set; Filtering and rearranging the search vectors in the intersection, and selecting the N search vectors with the highest accuracy; Wherein, M is a preset value.

4. The method for improving the retrieval accuracy of specific industry knowledge according to claim 2, characterized in that: The method further comprises: A plurality of the knowledge text blocks are generated according to the file database of the specific industry, and the knowledge text blocks are converted into vector representations and stored in the vector database.

5. The method for improving the retrieval accuracy of specific industry knowledge according to claim 4, characterized in that: The step of "generating a plurality of the knowledge text blocks according to the file database of the specific industry, and converting the knowledge text blocks into vector representations and storing them in the vector database" includes: For each file in the file database of the specific industry, read the text in the file and interpret the pictures in the file to generate a text file; Segment each of the text files to obtain a number of original text blocks that do not exceed a preset length; Performing text cleaning on the original text block based on the fourth language model to obtain the knowledge text block; Convert each of the knowledge text blocks into the corresponding vector representation; The mapping relationship between the knowledge text block and the vector representation is saved, and the vector representation is stored in the vector database.

6. The method for improving the retrieval accuracy of specific industry knowledge according to claim 5, characterized in that: The step of "segmenting each of the text files to obtain a plurality of original text blocks not exceeding a preset length" includes: Splitting each of the text files according to the first-level headings to obtain a plurality of first text blocks; Determine the length of each of the first text blocks, and if the length of a first text block exceeds the preset length, divide the first text block according to the secondary headings to obtain a plurality of second text blocks; Determine the length of each second text block, and if the length of a second text block exceeds the preset length, segment the second text block according to the third-level headings to obtain a plurality of third text blocks; Determine the length of each of the third text blocks, and if the length of a third text block exceeds the preset length, segment the third text block into paragraphs to obtain a plurality of fourth text blocks; Determine the length of each of the fourth text blocks, and if the length of a fourth text block exceeds the preset length, segment the fourth text block according to the period to obtain a plurality of fifth text blocks; Merging the adjacent fifth text blocks that are derived from the same fourth text block in order to obtain a plurality of sixth text blocks; each of the sixth text blocks contains one or more fifth text blocks, and the length does not exceed the preset length; Constructing a set of the original text blocks; The original text blocks include: the first text blocks, the second text blocks, the third text blocks, the fourth text blocks, the fifth text blocks that have not been merged, and the sixth text blocks, all of which have lengths not exceeding the preset length.

7. The method for improving the retrieval accuracy of specific industry knowledge according to claim 5, characterized in that: The method further comprises: Constructing a data set for the specific industry, wherein the data set includes a first preset number of question samples and corresponding standard answers; Dividing the data set into a training set and a test set according to a preset ratio; Using the training set and the LoRA technology, fine-tune a second preset number of large language models respectively; According to the input of the administrator, the first large language model, the second large language model, the third large language model and the fourth large language model are respectively selected from the second preset number of fine-tuned large language models, and the first large language model, the second large language model, the third large language model and the fourth large language model are the same or different.

8. The method for improving the retrieval accuracy of specific industry knowledge according to claim 7, characterized in that: The first prompt word is a preset prompt word, and the second prompt word and the third prompt word are preset prompt words or prompt words set by the administrator; The method further comprises: Using the test set to score the retrieval and generation effects based on the RAGAS standard, and obtaining a scoring result; The scoring results include: answer relevance, fidelity and context relevance.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method according to any one of claims 1 to 8.

10. A computer-readable storage device, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 8.

Images