Quality standard retrieval tool and method based on large language model

Abstract

The invention relates to a quality standard retrieval tool and method based on a large language model. The method comprises the following steps: storing a quality standard file, embedded vector data corresponding to the quality standard file and a mapping relation between the quality standard file and the embedded vector data into a quality standard file knowledge base; question vector data converted from a question described by a natural language is received, embedded vector data most similar to the question vector data is retrieved from a quality standard file knowledge base, and a quality standard file text corresponding to the embedded vector data is a related support document; recording the historical retrieval question and the corresponding retrieval result as historical question and answer information, and summarizing a preset response rule, the current retrieval question, the related support document and the historical question and answer information into cue words; and adopting a pre-trained large language model to generate a retrieval result according to the cue word.

Description

Technical Field

[0001] The present invention belongs to the technical field of aerospace systems, and relates to a quality standard retrieval tool and method based on large language models (LLM), which is applied to the research and development design process of aircraft, etc. Background Art

[0002] In the process of R&D and design of traditional aircraft, it is necessary to comply with the provisions of quality manuals and standard documents. With the accumulation of production experience, quality standards are constantly refined and updated. There may be problems such as inclusion and conflict between different quality standard documents for the same standard. In order to determine the most accurate parameter range, designers often need to consult and compare a large number of quality manuals and standard documents. The traditional retrieval method based on term frequency-inverse document frequency (TF-IDF) technology can only return specific text containing query keywords, and it is difficult to perform fuzzy retrieval for semantics and related fields. When the specific keywords are not determined, the efficiency of the search will be greatly reduced.

[0003] At the end of 2022, ChatGPT, a large language model product released by OpenAI, demonstrated multi-scenario, multi-purpose, and interdisciplinary task processing capabilities through natural language interaction. However, the reliability of the large language model cannot be effectively guaranteed. Although it has been trained with massive data and the content it generates is fluent and conforms to language rules and human values, it is insufficient in terms of factuality and timeliness. Therefore, although the large language model has the ability to answer questions across disciplines and fields in the design process and is suitable for fuzzy range retrieval of quality standard documents, further technical guarantees are needed for authenticity. Summary of the invention

[0004] The technical problem solved by the present invention is: in view of the business pain point that front-line personnel often need to consult a large number of quality manuals and standard documents during the R&D and design process, which reduces work efficiency, a quality standard retrieval tool and method based on a large language model is provided to achieve more flexible quality standard file retrieval, optimize the work process of consulting quality standards, and assist in improving design efficiency.

[0005] The solution to the technical problem of the present invention is: a quality standard retrieval tool based on a large language model, characterized by comprising a quality standard document knowledge base, a retrieval module, a prompt word generation module, and a response module;

[0006] A quality standard file knowledge base is used to store quality standard files and their corresponding embedded vector data, and a mapping relationship between quality standard files and embedded vector data;

[0007] The retrieval module receives a question described in natural language from a user, converts the question into embedded vector data, recorded as question vector data, performs similarity retrieval on the question vector data from the quality standard document knowledge base, that is, retrieves the embedded vector data most similar to the question vector data from the quality standard document knowledge base, and outputs the quality standard document text corresponding to the embedded vector data as a related supporting document to the prompt word generation module according to the mapping relationship between the quality standard document and the embedded vector data;

[0008] The prompt word generation module records the historical search questions and their corresponding search results as historical question and answer information, and summarizes the preset answer rules, current search questions, relevant supporting documents, and historical question and answer information as prompt words and sends them to the answer module;

[0009] The answer module uses a pre-trained large language model to generate retrieval results based on the prompt words and feeds the retrieval results back to the prompt word generation module.

[0010] Preferably, the above-mentioned quality standard retrieval tool based on large language model further includes a document loading module, a document segmentation module, and a vector conversion module;

[0011] The document loading module reads the full text in the quality standard file and forwards it to the document segmentation module;

[0012] The document segmentation module segments the full text in the quality standard file into sliced ​​texts; the sliced ​​texts contain complete semantics, and the number of characters does not exceed the preset number of slice words;

[0013] The vector conversion module maps the sliced ​​text to a high-dimensional semantic vector space to obtain embedded vector data of fixed length, and stores the embedded vector data in the quality standard file knowledge base.

[0014] Preferably, the quality standard file is a word file, a pdf file or an xml file.

[0015] Preferably, the document loading module selects a corresponding loading method according to the type of the quality standard file, including reading character text from word and xml files, and performing optical character recognition reading on scanned pdf files.

[0016] Preferably, the segmentation method of the document segmentation module is: first, segment according to the underlying title, and group the texts belonging to the same title into one slice text in the order from the front to the back of the document; then, according to the size of the slice text, the text with a character number greater than the preset slice word number is further divided according to the preset slice word number, and there is a certain number of character overlaps at the beginning and end of adjacent slice texts.

[0017] Preferably, the weight of the large language model is quantized using a non-saturated quantization method, and the input and output of the activation function of the large language model are quantized using a saturated quantization method.

[0018] Preferably, the similarity retrieval method is to calculate the cosine distance of the embedded vector data as the similarity, and select the N embedded vector data with the greatest similarity after sorting them by size.

[0019] Preferably, the preset response rules include the following:

[0020] If multiple quality standard documents have an inclusion relationship with respect to the same standard provision, the large language model is prompted to return the intersection of the scopes of the standard provisions in the multiple quality standard documents when responding; if multiple quality standard documents have a conflicting relationship with respect to the same standard provision, the large language model is prompted to return all the contents of the standard provisions in each quality standard document when responding.

[0021] Preferably, multiple quality standard document knowledge bases are established to store vector data corresponding to quality standard documents of different sensitivity levels, perform data isolation, divide different user groups, manage access rights, and the retrieval module performs similarity retrieval from the corresponding quality standard document knowledge base based on the user's authority.

[0022] Another technical solution of the present invention is: a quality standard retrieval method based on a large language model, the method comprising the following steps:

[0023] The quality standard file and its corresponding embedded vector data, and the mapping relationship between the quality standard file and the embedded vector data are stored in a quality standard file knowledge base;

[0024] Receive a question described in natural language input by a user, convert the question into embedded vector data, record it as question vector data, perform similarity search on the question vector data from a quality standard document knowledge base, and retrieve N embedded vector data that are most similar to the question vector data from the quality standard document knowledge base;

[0025] According to the mapping relationship between the quality standard file and the embedded vector data, the quality standard file texts corresponding to N embedded vector data are obtained as relevant supporting documents, where N is greater than or equal to 3;

[0026] Record historical search questions and their corresponding search results as historical question-and-answer information, and summarize preset answer rules, current search questions, relevant supporting documents, and historical question-and-answer information as prompt words;

[0027] A pre-trained large language model is used to generate search results based on prompt words. Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) The present invention constructs a comprehensive knowledge base for quality standard documents, uses a large language model and combines the question-answering mechanism of natural language processing to conduct in-depth analysis and accurate answers to the document content.

[0029] (2) The present invention uses a vector database tool to effectively manage the generated text vectors, thereby realizing an efficient text retrieval function.

[0030] (3) The present invention interprets the user input questions described in natural language through the vector data similarity retrieval method and the prompt engineering method to obtain supporting documents, thereby supporting efficient and flexible internal knowledge modification and accurate content generation;

[0031] (4) The large prediction model of the present invention is a quantized model, which reduces the amount of calculation and memory usage, and can be deployed on mobile and embedded devices;

[0032] (5) The present invention proposes a method of using a separately deployed vector database to achieve autonomous control and adjustment of the data source for large-scale language model retrieval enhancement generation, thereby ensuring data security and supporting isolated management of data in different fields and with different levels of sensitivity.

[0033] (6) The present invention alleviates the hallucination problem of the large language model by injecting preset response rules as prompt words into the large language model, so that the generated content can be based on evidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a workflow diagram of a quality standard retrieval tool based on a large language model according to an embodiment of the present invention.

[0035] Figure 2 This is the vector database workflow of an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with the embodiments.

[0037] like Figure 2 As shown, the present invention provides a quality standard retrieval tool and method based on a large language model. The overall technical solution of the retrieval tool is: using the basic language understanding and text generation capabilities of the pre-trained large language model, plus a vector database to archive and retrieve text materials. The tool includes a document loading module, a document segmentation module, a vector conversion module, a quality standard document knowledge base, a retrieval module, a prompt word generation module, and a response module;

[0038] The document loading module reads the full text in the quality standard file and forwards it to the document segmentation module;

[0039] The document segmentation module segments the full text in the quality standard file into sliced ​​texts; the sliced ​​texts contain complete semantics, and the number of characters does not exceed the preset number of slice words;

[0040] The vector conversion module maps the slice text to a high-dimensional semantic vector space to obtain embedded vector data of fixed length, and stores the embedded vector data in the quality standard file knowledge base;

[0041] A quality standard file knowledge base is used to store quality standard files and their corresponding embedded vector data, and a mapping relationship between quality standard files and embedded vector data;

[0042] The retrieval module receives a question described in natural language from a user, converts the question into embedded vector data, recorded as question vector data, performs similarity retrieval on the question vector data from the quality standard document knowledge base, that is, retrieves N embedded vector data that are most similar to the question vector data from the quality standard document knowledge base, and outputs the quality standard document texts corresponding to the N embedded vector data as relevant supporting documents to the prompt word generation module according to the mapping relationship between the quality standard document and the embedded vector data, where N is greater than or equal to 3;

[0043] The prompt word generation module records the historical search questions and their corresponding search results as historical question and answer information, and summarizes the preset answer rules, current search questions, relevant supporting documents, and historical question and answer information as prompt words and sends them to the answer module;

[0044] The answer module uses a pre-trained large language model to generate retrieval results based on the prompt words and feeds the retrieval results back to the prompt word generation module.

[0045] Specifically, quantization technology is used to accelerate the deployment of large models on consumer-grade graphics cards, vector database technology is used in the document retrieval process, and prompt engineering technology is applied to the summary and transmission of documents. The implementation plan for applying these technologies is explained below.

[0046] (1) Quantitative technology

[0047] Model quantization refers to converting a floating-point model into a fixed-point model through some method. For example, the original float32 model weight parameters are quantized into int8 type. This is used to reduce the amount of calculation and memory usage, and solve the problem of deploying inference of deep learning models on mobile and embedded devices. However, it will inevitably bring about a certain loss of accuracy.

[0048] Specifically, the present invention adopts an asymmetric quantization method of linear quantization in quantization technology, and the calculation process is shown in the following formula:

[0049]

[0050] Among them, Q is the quantized data, R is the data before quantization, S is the scaling factor, Z is the bias factor, T max is the cutoff threshold.

[0051] The weight of the large language model of the present invention is quantized by a non-saturated quantization method, and the input and output of the activation function of the large language model are quantized by a saturated quantization method.

[0052] (2) Vector database technology

[0053] like Figure 2 As shown in the figure, vector database is also called vector database, which is mainly used to store and process vector data. Compared with traditional relational database, vector database can convert unstructured data (images, texts and videos, etc.) into vector data for storage through some transformation or embedding, so as to realize similarity retrieval. Different from the traditional database query method based on exact matching, vector database can realize relevant search based on contextual semantics. Its main feature is efficient storage and retrieval. The use of index technology and vector retrieval algorithm can realize fast response under high-dimensional big data.

[0054] The quality standard file is a word file, a pdf file or an xml file.

[0055] The document loading module selects a corresponding loading method according to the type of the quality standard file, including reading character text from word and xml files, and performing optical character recognition reading on scanned pdf files.

[0056] The segmentation method of the document segmentation module is: first, segment according to the title, and classify the text belonging to the same title into a slice text in the order from the front to the back of the document; then, according to the size of the slice text, the text with a character number greater than the preset slice word number is further divided according to the preset slice word number, and there is a certain number of character overlaps at the beginning and end of adjacent slice texts.

[0057] The present invention inputs a vector (problem vector data) and searches for N vectors that are most similar to the input vector from the vector database and returns them. In practical applications, it is also necessary to save the mapping relationship between data and vectors, that is, to return N vectors and their data ids, and obtain specific data according to the data ids, as shown in the attached figure. Figure 1 shown.

[0058] The similarity retrieval method is to calculate the cosine distance of the embedded vector data as the similarity, sort them by size and select the N embedded vector data with the greatest similarity.

[0059] (3) Prompt engineering technology

[0060] Proper prompt words can improve the ability of large language models to handle complex task scenarios. Prompt engineering plays an important role in interacting and connecting with large language models and understanding their capabilities.

[0061] In actual practice, there are two main purposes for using prompt words. One is to obtain specific results for specific problems, and the other is to solidify the prompt words into the program to become part of the system function. The quality standard retrieval based on the vector database of the present invention is the second one.

[0062] The main components of the prompt words include: 1) instructions, which describe the task; 2) context, which gives other background information related to the task; 3) examples, which give examples when necessary, which is called few-shot learning or contextual learning in academia. Practice has proved that it is helpful for output correctness; 4) input, the input information of the task, which needs to be clearly identified in the prompt words; 5) output, which describes the format of the output so that subsequent modules can automatically parse the output results of the model, such as JSON, XML format, etc.

[0063] The present invention accepts a problem description and retrieves a set of relevant supporting documents. These relevant supporting documents are combined as context and original input prompt words and passed to a large language model as a text generator to obtain the final output, supporting efficient and flexible internal knowledge modification and accurate content generation.

[0064] In view of the possible inclusion and conflict between different standard documents regarding the same standard, the response rules are used to limit the response of the large language model through the prompt word generation module. The preset response rules include the following:

[0065] If multiple quality standard documents have an inclusion relationship with respect to the same standard provision, the large language model is prompted to return the intersection of the scopes of the standard provisions in the multiple quality standard documents when responding; if multiple quality standard documents have a conflicting relationship with respect to the same standard provision, the large language model is prompted to return all the contents of the standard provisions in each quality standard document when responding.

[0066] (4) Private deployment vector database

[0067] In order to establish multiple quality standard document knowledge bases to store the vector data corresponding to quality standard documents of different sensitivity levels, isolate the data, divide different user groups, and manage access rights, the retrieval module performs similarity retrieval from the corresponding quality standard document knowledge base according to the user's permissions.

[0068] Based on the above tool, the present invention also provides a quality standard retrieval method based on large language model retrieval-augmented generation technology (RAG), which includes the following steps:

[0069] The quality standard file and its corresponding embedded vector data, and the mapping relationship between the quality standard file and the embedded vector data are stored in a quality standard file knowledge base;

[0070] Receive a question described in natural language input by a user, convert the question into embedded vector data, record it as question vector data, perform similarity search on the question vector data from a quality standard document knowledge base, and retrieve the embedded vector data most similar to the question vector data from the quality standard document knowledge base;

[0071] According to the mapping relationship between the quality standard file and the embedded vector data, the quality standard file text corresponding to the embedded vector data is obtained as a related supporting document;

[0072] Record historical search questions and their corresponding search results as historical question-and-answer information, and summarize preset answer rules, current search questions, relevant supporting documents, and historical question-and-answer information as prompt words;

[0073] Use a pre-trained large language model to generate retrieval results based on the prompt words.

[0074] In summary, the present invention has deeply optimized functions such as document parsing, text segmentation, data reflow, and large model thinking chain to better adapt to the actual business scenarios of quality standard retrieval, ensure accurate and efficient extraction of required text information from various files, and be able to accurately understand and process user input, provide high-quality output results, thereby improving the user experience of the system.

[0075] The present invention can be used to assist in improving the work efficiency of R&D and design personnel, and can also be used as an office assistant to provide functions such as data analysis and report writing, saving the time and energy of front-line personnel and improving overall office efficiency.

[0076] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A quality standard retrieval tool based on a large language model, characterized by It includes a quality standard document knowledge base, a retrieval module, a prompt word generation module, and a response module; A quality standard file knowledge base is used to store quality standard files and their corresponding embedded vector data, and a mapping relationship between quality standard files and embedded vector data; The retrieval module receives a question described in natural language from a user, converts the question into embedded vector data, recorded as question vector data, performs similarity retrieval on the question vector data from the quality standard document knowledge base, that is, retrieves the embedded vector data most similar to the question vector data from the quality standard document knowledge base, and outputs the quality standard document text corresponding to the embedded vector data as a related supporting document to the prompt word generation module according to the mapping relationship between the quality standard document and the embedded vector data; The prompt word generation module records the historical search questions and their corresponding search results as historical question and answer information, and summarizes the preset answer rules, current search questions, relevant supporting documents, and historical question and answer information as prompt words and sends them to the answer module; The answer module uses a pre-trained large language model to generate retrieval results based on the prompt words and feeds the retrieval results back to the prompt word generation module.

2. A quality standard search tool based on a large language model according to claim 1, characterized in that It also includes a document loading module, a document segmentation module, and a vector conversion module; The document loading module reads the full text in the quality standard file and forwards it to the document segmentation module; The document segmentation module segments the full text in the quality standard file into sliced ​​texts; the sliced ​​texts contain complete semantics, and the number of characters does not exceed the preset number of slice words; The vector conversion module maps the sliced ​​text to a high-dimensional semantic vector space to obtain embedded vector data of fixed length, and stores the embedded vector data in the quality standard file knowledge base.

3. According to claim 1, a quality standard search tool based on a large language model is characterized in that: The quality standard file is a word file, a pdf file or an xml file.

4. A quality standard search tool based on a large language model according to claim 3, characterized in that The document loading module selects a corresponding loading method according to the type of the quality standard file, including reading character text from word and xml files, and performing optical character recognition reading on scanned pdf files.

5. The quality standard search tool based on a large language model according to claim 1, characterized in that: The segmentation method of the document segmentation module is as follows: first, segment according to the underlying title, and group the texts under the same title into a slice text in the order from the front to the back of the document; then, according to the size of the slice text, the text with a character number greater than the preset slice word number is further divided according to the preset slice word number, and there is a certain number of character overlaps at the beginning and end of adjacent slice texts.

6. The quality standard search tool based on a large language model according to claim 1, characterized in that: The weight of the large language model is quantized using a non-saturated quantization method, and the input and output of the activation function of the large language model are quantized using a saturated quantization method.

7. The quality standard search tool based on a large language model according to claim 1, characterized in that: The similarity retrieval method is to calculate the cosine distance of the embedded vector data as the similarity, and select the N embedded vector data with the greatest similarity after sorting them by size.

8. The quality standard search tool based on a large language model according to claim 1, characterized in that: The preset answering rules include the following: If multiple quality standard documents have an inclusion relationship with respect to the same standard provision, the large language model is prompted to return the intersection of the scopes of the standard provisions in the multiple quality standard documents when responding; if multiple quality standard documents have a conflicting relationship with respect to the same standard provision, the large language model is prompted to return all the contents of the standard provisions in each quality standard document when responding.

9. The quality standard search tool based on a large language model according to claim 1, characterized in that: Establish multiple quality standard document knowledge bases to store vector data corresponding to quality standard documents of different sensitivity levels, isolate data, divide different user groups, manage access rights, and the retrieval module performs similarity retrieval from the corresponding quality standard document knowledge base based on user permissions.

10. A quality standard retrieval method based on a large language model, characterized in that The steps include: The quality standard file and its corresponding embedded vector data, and the mapping relationship between the quality standard file and the embedded vector data are stored in a quality standard file knowledge base; Receive a question described in natural language input by a user, convert the question into embedded vector data, record it as question vector data, perform similarity search on the question vector data from a quality standard document knowledge base, and retrieve N embedded vector data that are most similar to the question vector data from the quality standard document knowledge base; According to the mapping relationship between the quality standard file and the embedded vector data, the quality standard file texts corresponding to N embedded vector data are obtained as relevant supporting documents, where N is greater than or equal to 3; Record historical search questions and their corresponding search results as historical question-and-answer information, and summarize preset answer rules, current search questions, relevant supporting documents, and historical question-and-answer information as prompt words; Use a pre-trained large language model to generate retrieval results based on the prompt words.

Images